La evolución de

los insights desde la

escucha social a la

lectura por imagen:

El caso L’Oreal

Evolving Social

Insights from E-Listening

to E-Seeing:

The L’Oréal Case

aDResearch ESIC

Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29

Yagüez Lorenzo, E. y Rodríguez Romo, A. (2020)

Evolving Social Insights from E-Listening to E-Seeing:

The L’Oréal Case

Revista Internacional de Investigación en Comunicación

aDResearch ESIC. Nº 23 Vol 23

Segundo semestre, julio-diciembre 2020 · Págs. 8 a 29

https://doi.org/10.7263/adresic-023-01

Estefanía Yagüez Lorenzo

Market Intelligence and Consumer Insight Director

L’Oréal

estefania.yaguez@loreal.com

Alberto Rodríguez Romo

Market & ROI Insights Leader

L’Oréal

Alberto.RODRIGUEZROMO@loreal.com

Purpose: Show how L’Oréal Spain evolved from a classic social listening practice, to image

driven insights by the usage of AI tools; the rst step to correctly understand the vast amount of

personal information, interests and desires that millions of consumers share every day in social

networks.

Design / Methodology / Approach: Hybrid approach, combining AI to quickly categorize, lter

and run a preliminary analysis on images, with a manual coding of specic elements about the

beauty industry. This procedure achieved two signicant goals:

• First-hand learnings that could be directly used by marketing teams,

• A deep coded beauty image database that could be used to train AI algorithms in the

future for a more powerful, automatized tool.

Results: There are seven main tribes within the makeup universe in Spain, based on their tech-

niques and products used. Furthermore, its evolution was also traced, and measured within

customers, inuencers and brands, to detect where a ‘sweet spot’ was taking place: a rising

trend that was growing fast among consumers and inuencers, but where brands were not

yet present.

Limitations / Implications: The main limitations of the study are two. On the one hand, the

representativeness of Instagram for the Spanish beauty consumer. It was necessary to weight

the dierent segments of the sample to reach the right, balanced conclusions. And on the

other, AI algorithms capacity to code and lter images. At a certain stage, it was necessary to

introduce human help to validate and perform a deeper coding.

Contribution: Setting the rst milestone on social image insights for the beauty industry,

advancing from a purely text-based insight practice to a much richer information based on the

looks that consumers share every day.

RESUMEN

ABSTRACT

Clasicación JEL:

M31

Palabras clave:

Escucha social,

lectura por imagen,

inteligencia articial,

aprendizaje

automático,

reconocimiento

facial

JEL Classication:

M31

Key words:

Social listening,

e-seeing,

articial intelligence,

machine learning,

facial recognition

Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29

Objetivo: Mostrar la evolución de L’Oréal España desde un modelo tradicional de escucha

social basado en texto, al análisis de imágenes a través de herramientas de Inteligencia Arti-

cial (IA); el primer paso para entender la ingente cantidad de información personal, intereses y

deseos que millones de consumidores comparten cada día en redes sociales.

Diseño/metodología/enfoque: Enfoque híbrido, combinando IA para la rápida categoriza-

ción, ltrado y análisis de imágenes, junto con una categorización manual de características

especícas sobre la belleza. Esta metodología permitió alcanzar dos metas signicativas:

• Aprendizajes preliminares para los equipos de marketing, de aplicación inmediata

• Base de datos especíca en belleza con información muy detallada, para «entrenar» futu-

ros algoritmos de IA y crear herramientas más potentes y automatizadas.

Resultados: Existen siete principales tribus en el universo del maquillaje en España, basándonos

en las técnicas y productos que utilizar. También se midió su evolución en grupos indepen-

dientes de consumidores, inuencers y marcas, para detectar las mejores oportunidades de

inversión y tendencias en auge.

Implicaciones/limitaciones: Las principales limitaciones del estudio son dos. Por un lado, la

representatividad de Instagram sobre la consumidora de belleza en España. Es necesario pon-

derar las diferentes celdas de la muestra para alcanzar las conclusiones correctas y balanceadas.

Y por otro lado, la capacidad de los algoritmos de IA para codicar y ltrar imágenes. En un

determinado punto, es necesaria la interacción humana para validar y codicar en mayor pro-

fundidad los resultados preliminares.

Originalidad/contribución: Interpretación de imágenes en redes sociales para la industria de

la belleza, avanzando desde el análisis de textos a una información mucho más rica y basada

en imágenes que las consumidoras comparten en redes sociales.

aDResearch ESIC

Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29

the forums and blogs. As a consequence of this

preeminence of the text in the origins of the So-

cial Web, both in the academic eld and in the

industry itself, much more effort and resources

have been devoted to text mining, in general,

and to semantic analysis, in particular, than to

image analysis. In fact, the sharpness of the se-

mantic analysis was the key element of these

tools, ultimately determining their potential.

Not an easy task, actually: often sarcastic ex-

pressions, acronyms, passive voice expressions

and other related verbalizations were not always

correctly identied by the tools and categorized

incorrectly as a ‘positive’ or ‘negative’ reaction

being the opposite case. These tools have al-

ways required a strong database to start with,

constant updates and to a very large extent, hu-

man supervision and ne tuning (Romero &

Gil, 2008).

At the end of the last decade, two great tech-

nological revolutions occurred that would end

the hegemony of the text on the Social Web. On

the one hand, the penetration of the mobile Inter-

net radically increased, bringing connectivity to a

device equipped with a photographic camera. In

parallel, Facebook reaches the “Tipping Point”,

thanks in turn to a large extent to the develop-

ment of the Mobile Internet. Facebook is the rst

massive social network that facilitates and pro-

motes communication between users based on

images (mostly photos). A few years later, Twitter,

which was born as a social network oriented to

written communication, would migrate towards

audiovisual communication. Later on, Instagram

emerged, giving the text an almost residual role,

purely related to hashtags for visibility, topic

search and connections (IAB Spain, 2018).

While these transformations were taking

place, the “Social Listening” industry and moni-

toring tools stayed anchored in the “textual” dy-

1. Background: From social

listening to e-seeing

The interest of the market research industry for

what consumers share in digital media begins

with the launch of the so-called Web 2.0, which

put in the hands of almost any user the possibili-

ty of expressing themselves through these plat-

forms without the technical and economical bar-

riers that hindered the democratization of this

practice. Suddenly, millions of users began to

share their opinions and experiences related to

consumption and brands, which was quickly

perceived by the industry as an opportunity to

complete consumer knowledge obtained through

traditional sources (surveys, focus group, con-

sumer panels, etc.). Rather than sampled con-

sumers, for the rst time it was possible to listen

to the true, unltered consumer opinion on a real

time basis (Celaya, 2008).

Almost at the same time, a parallel industry,

that of “Social Listening”, was also developed,

led by small technology companies, which put

on the market tools that allowed the collection

and analysis of what was published on the Social

Web from text-based queries. After setting one

or more keywords, these tools connect to the da-

tabases of social networks, via API, extract the

contents in which there is a match with those

keywords, store them and analyze them, both

quantitatively and qualitatively. In short, these

are tools whose main value for the consumer re-

search industry lies in analyzing “what is writ-

ten”, the word.

The development of commercial Social Lis-

tening tools coincides with a period (mid-last

decade) in which the Social Web was dominat-

ed by platforms in which the text had a clearly

greater weight than the image, as was the case of

1 Thanks to: Nethodology & Microsoft

Evolving Social Insights from E-Listening to E-Seeing: The L’Oréal Case · págs. 8 a 29

namics of the origins of the Social Web. This

meant not only a loss of lots of information that

consumers were sharing in social networks, but

also a real challenge for sectors, such as beauty

or fashion, in which the user is constantly com-

municating their tastes and preferences in these

consumer categories, even when the message

they want to convey has nothing to do with it. In

practically any photo uploaded to Instagram in

which a person appears, there will almost cer-

tainly be an outt, a hairstyle and a facial look

(even if it is a completely clean face). Suddenly,

missing out your consumers’ communications

in channels that were so disruptive had become

a risk for brands. If you were not able to be in

touch with your consumers in a moment when

they are broadly sharing most of their private life

publicly, you could be perceived as an out of

date brand, losing the connection with your au-

dience. (Borgatti,Everett &Johnson, 2018).

By the late 2000’s, some technological revolu-

tions disrupted the social networks universe.

Firstly, smartphones were massively adopted,

giving online access to users equipped with in-

creasingly better cameras. In parallel, Facebook

drastically switched its philosophy to a mobile,

more image-based network; as an example, to

illustrate this strategy mindset, they even had

‘mobile only days in the Facebook HQ, where all

laptops were shut down and it was only possible

to work via mobile-. Later on, Twitter enabled

better visual content on their feeds. Finally, Ins-

tagram disrupted the market with a pure im-

age-based concept, with very little importance

given to text. Due to high user engagement rate,

Instagram also a valuable social media market-

ing tool. The most important consumer sociali-

zation variables are with an emphasis on peer

communications, brand related factors, and the

use of the social network for eWOM among the

users of Instagram. When consumers follow

brands in a social network, the brands may cre-

ate their desirable attitudes and build loyalty in

consumers. In addition, engagement through

social media platforms can play a very important

role in building brand relationship quality. By

studying how users imitate the behaviour of oth-

er users and their attitudes towards brands, they

can distinguish the users’ perception of relation-

ship quality, their use of social networks, the

number of brands that they follow, and the indi-

viduals who participate in brand-related eWOM.

This distinction can help marketers discover the

most likely perspective of brand management.

(Delafrooz, Rahmati & Abdi, 2019).

Facebook quickly perceived this emerging

trend and purchased the booming app for the

sum of $ 1Bn in April 2012. Considered a rip off

by then, now Facebook’s stock is worth four

times its 2012 average price —and comple-

menting its mature founding social network

with a fresh, younger and wildly image based

platform has denitely played a big role into it.

Not only image is trending, but it is also de-

manded by youngsters— the most intensive users

of social networks (Borgatti et al., 2018).

Furthermore, in the following years, all dis-

ruptive approaches in the social network market

—such as Snapchat stories, which were later

replicated by Instagram— were purely visual.

The latest trend is Tik Tok, a 100 % video-based

network based on short interactions from three

to 60 seconds. Consumers happily embraced

this shift and massively adopted the image-based

communication, therefore limiting the room for

precise text analytics. In a context where an im-

age tells a thousand words, text is only used to

mention, link, or cheer up, rather than provid-

ing an insightful context about a consumer’s feel-

ings or likes.

aDResearch ESIC

Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29

Lots of visual information with key insights

was being missed due to social listening limita-

tions. In this situation, moving from text-based

social listening to image-based, or e-seeing, was

not only a logical next step, but also an absolute

need to continue to be in touch with consumers.

As the world industry leader, it is a must for

L’Oréal to understand how people connect, en-

gage, share and enjoy their beauty across ages,

genders and likes, helping marketers to match

the right audiences, in the right moment, with

the right content, through the right channels

(Borgattiet al., 2018).

The very name of the technique— Social Lis-

tening— already hints that the focus is on the

text; an example of a typical output can be seen

on Figure 1. You “listen” to what is “said”, what is

“written”. As it has been exposed, this approach

leaves out something tat in consumer research is

essential: “what is done”, which, transferred to

the context of social networks, would be “what

is shown”. “We listen”, but we do not “see”

(Ramos et al., 2020).

In conventional qualitative research (through

ethnography, focus group, in-depth interviews,

etc.) the advantages of image analysis are well

known. They are used, for example, to allow

consumers to express ideas that are repressed

(consciously or unconsciously) or that are so

complex that people nd it difcult to verbalize

them. They are also used for the user to project

(as in the collage technique). And, of course,

they play a fundamental role in ethnographic re-

search by providing context information that

hardly appears when other forced context tech-

niques are used, such as focus groups (Malhotra,

1996; Silverman, 2016).

These same advantages are fully extrapolated

to research on social networks. There are mil-

lions of users who have stopped talking to start

showing. It may be asked then why even today

the analysis of images published on social net-

works continues to be marginal. It is necessary

to adopt new tools and develop methodologies

that incorporate this source to enrich analyses

and obtain better insights. In short, move from

Social Listening to e-watching.

Now, more than ever before, ethnographers are

using visual and digital images and technologies to

research and represent the cultures, lives and ex-

periences of other people. Theoretical and techno-

logical innovations have made the visual both ac-

ceptable and accessible to anthropologists and this

has created a contemporary context where new

ethnographic media, methodologies and practices

are emerging (Alfonso, Kurti & Pink, 2004).

Articial intelligence, a great ally. Internet

monitoring tools have begun to take some timid

Figure 1. Classic text query results and topic analysis

Evolving Social Insights from E-Listening to E-Seeing: The L’Oréal Case · págs. 8 a 29

steps for tracing images in which an object of

interest for research appears. Several of the most

relevant tools in this market already have func-

tionalities that allow, for example, detecting im-

ages in which a brand logo appears, even if it is

not mentioned in text.

These advances, still clearly insufcient, have

been possible thanks to the technological boost

given to image recognition by Internet big play-

ers, with Google, Amazon or Microsoft leading

the way. Their need to analyze, classify and

moderate the vast amount of images published

every minute on the Internet is behind the

speed with which these technologies, support-

ed by articial intelligence, have been devel-

oped and democratized. And of their own need,

these companies have found a new market

niche that involves making these technologies

available to third parties. Google has Cloud Vi-

sion, a service via API, which allows to extract

information from images, such as a wide catalog

of objects, recognition of logos or faces, and cre-

ate new custom models.

With time, these tools further developed and

enabled a series of relevant insights, such as in-

corporating objective information from the us-

ers’ proles in social networks, or improving the

semantic analyses to understand the consumers’

sentiment towards a product - whether positive,

negative or neutral, as appreciated on example

Figure 2. This was particularly relevant for un-

derstanding customers’ proles, to manage spe-

cic crises or to analyse the brand engagement

through time, although again, purely based on

text interactions (Alfonso et al., 2004).

As above mentioned, the limitations of these

tools are considerable, while the possibilities

that e-seeing offers are huge. And not only in

terms of insight, but also on scalability. A lan-

guage processing tool faces constraints when

switching to a different language (or even be-

tween different language regions, as Spain or

Latin America could be), and needs to be thor-

oughly reviewed. On the other hand, image pro-

cessing has the enormous power to be automat-

ically escalated to any other country with very

minor tweaks. If you can train a tool to detect

and lter a specic element in an image, that el-

ement can be traced in any picture taken around

the world; now imagine the potential that this

can have applied to a global scale. In the case of

L’Oréal, it can be possible to leverage local devel-

opments, and a small initiative can completely

change the way the entire organizations relates

with their customers in a relatively short period

of time – denitely shorter than the usual tim-

ings needed for long-scale organizational chang-

es (Kaneshige, 2015).

Figure 2. Gender, interest and sentiment analysis

aDResearch ESIC

Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29

L’Oréal itself is an example on how im-

age-based IA can be easily escalated, not only

geographically, but also in the way that a simi-

lar algorithm (identifying objects in an image)

can be utilized for different functions. The fol-

lowing tools were recently deployed within the

L’Oréal insights community and have helped

the teams since:

• Deep Vision (Figures 3-4): Image recogni-

tion and linkage to other similar ones on

Instagram, enabling the classication of

over 50,000 images into different clusters

of similarity and tracing of its brand. Visu-

alization shown below.

This tool enables L’Oréal teams to monitor the

key trends and competitor activity, as well as ad-

Figure 3. Deep Vision make-up landing screen

Figure 4. Deep Vision zoom on eye make-up and NYX professional make-up

Evolving Social Insights from E-Listening to E-Seeing: The L’Oréal Case · págs. 8 a 29

just campaign investment weights based on their

axis strategy. It is also used to visualise examples

around given trends.

• Golden Eye (Figure 5): the same algorithm,

but focused on quick and sharp image se-

lection for marketing teams. This tool high-

lights the most relevant elements of a pic-

ture and contrasts them with Microsoft’s

and MIT’s database of neuronal studies:

over 70,000 cases in total to match with

our algorithm, to predict image salience

and memorability. This way, marketing

teams can quickly make choices between

multiple ambassadors, creativities, product

images and image compositions, with a sol-

id criteria behind and with the help of AI.

2. The approach of L’Oréal Insights:

Partnering with AI

When rst facing the challenge of making this

relevant shift, the L’Oréal Consumer & Market

Intelligence team searched for inspiration among

well known traditional techniques. Additionally,

Figure 5. GoldenEye interface and results

in the same way images were a great support for

conventional qualitative techniques, the goal

was to develop similar capabilities, but with a

more ambitious scope. In traditional qualitative

research, showing images directly to consumers

is often a good approach for gathering rst-

hand, direct reactions, but it is also useful to use

projection techniques to dig deeper into con-

sumers’ feelings, and understand the underlying

perceptions that matter the most - the true in-

sight. Translating this philosophy into e-seeing,

the goal was to not only perceive the evident,

but also spot the trend, identify the tribe and

separate trend-setters from the followers. At the

opposite end with a pure tech approach, AI was

taking its rst steps into the social listening in-

dustry. The rst tools enabled logo identication

within sets of images, or basic information: gen-

der, age, etc. Gradually, big players entered the

market, such as Google with Cloud Vision, or

Microsoft with Computer Vision. These were

more powerful tools, enabling a deeper analysis

for the Consumer & Market Intelligence teams

(Mulfari, 2016; Del Sole, 2017).