La evolución de
los insights desde la
escucha social a la
lectura por imagen:
El caso LOreal
Evolving Social
Insights from E-Listening
to E-Seeing:
The LOréal Case
aDResearch ESIC
Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29
Yagüez Lorenzo, E. y Rodríguez Romo, A. (2020)
Evolving Social Insights from E-Listening to E-Seeing:
The L’Oréal Case
Revista Internacional de Investigación en Comunicación
aDResearch ESIC. Nº 23 Vol 23
Segundo semestre, julio-diciembre 2020 · Págs. 8 a 29
Estefanía Yagüez Lorenzo
Market Intelligence and Consumer Insight Director
Alberto Rodríguez Romo
Market & ROI Insights Leader
Purpose: Show how L’Oréal Spain evolved from a classic social listening practice, to image
driven insights by the usage of AI tools; the rst step to correctly understand the vast amount of
personal information, interests and desires that millions of consumers share every day in social
Design / Methodology / Approach: Hybrid approach, combining AI to quickly categorize, lter
and run a preliminary analysis on images, with a manual coding of specic elements about the
beauty industry. This procedure achieved two signicant goals:
First-hand learnings that could be directly used by marketing teams,
A deep coded beauty image database that could be used to train AI algorithms in the
future for a more powerful, automatized tool.
Results: There are seven main tribes within the makeup universe in Spain, based on their tech-
niques and products used. Furthermore, its evolution was also traced, and measured within
customers, inuencers and brands, to detect where a ‘sweet spot was taking place: a rising
trend that was growing fast among consumers and inuencers, but where brands were not
yet present.
Limitations / Implications: The main limitations of the study are two. On the one hand, the
representativeness of Instagram for the Spanish beauty consumer. It was necessary to weight
the dierent segments of the sample to reach the right, balanced conclusions. And on the
other, AI algorithms capacity to code and lter images. At a certain stage, it was necessary to
introduce human help to validate and perform a deeper coding.
Contribution: Setting the rst milestone on social image insights for the beauty industry,
advancing from a purely text-based insight practice to a much richer information based on the
looks that consumers share every day.
Clasicación JEL:
Palabras clave:
Escucha social,
lectura por imagen,
inteligencia articial,
JEL Classication:
Key words:
Social listening,
articial intelligence,
machine learning,
facial recognition
Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29
Objetivo: Mostrar la evolución de L’Oréal España desde un modelo tradicional de escucha
social basado en texto, al análisis de imágenes a través de herramientas de Inteligencia Arti-
cial (IA); el primer paso para entender la ingente cantidad de información personal, intereses y
deseos que millones de consumidores comparten cada día en redes sociales.
Diseño/metodología/enfoque: Enfoque híbrido, combinando IA para la rápida categoriza-
ción, ltrado y análisis de imágenes, junto con una categorización manual de características
especícas sobre la belleza. Esta metodología permitió alcanzar dos metas signicativas:
Aprendizajes preliminares para los equipos de marketing, de aplicación inmediata
Base de datos especíca en belleza con información muy detallada, para «entrenar» futu-
ros algoritmos de IA y crear herramientas más potentes y automatizadas.
Resultados: Existen siete principales tribus en el universo del maquillaje en España, basándonos
en las técnicas y productos que utilizar. También se midió su evolución en grupos indepen-
dientes de consumidores, inuencers y marcas, para detectar las mejores oportunidades de
inversión y tendencias en auge.
Implicaciones/limitaciones: Las principales limitaciones del estudio son dos. Por un lado, la
representatividad de Instagram sobre la consumidora de belleza en España. Es necesario pon-
derar las diferentes celdas de la muestra para alcanzar las conclusiones correctas y balanceadas.
Y por otro lado, la capacidad de los algoritmos de IA para codicar y ltrar imágenes. En un
determinado punto, es necesaria la interacción humana para validar y codicar en mayor pro-
fundidad los resultados preliminares.
Originalidad/contribución: Interpretación de imágenes en redes sociales para la industria de
la belleza, avanzando desde el análisis de textos a una información mucho más rica y basada
en imágenes que las consumidoras comparten en redes sociales.
aDResearch ESIC
Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29
the forums and blogs. As a consequence of this
preeminence of the text in the origins of the So-
cial Web, both in the academic eld and in the
industry itself, much more effort and resources
have been devoted to text mining, in general,
and to semantic analysis, in particular, than to
image analysis. In fact, the sharpness of the se-
mantic analysis was the key element of these
tools, ultimately determining their potential.
Not an easy task, actually: often sarcastic ex-
pressions, acronyms, passive voice expressions
and other related verbalizations were not always
correctly identied by the tools and categorized
incorrectly as a ‘positive’ or ‘negative’ reaction
being the opposite case. These tools have al-
ways required a strong database to start with,
constant updates and to a very large extent, hu-
man supervision and ne tuning (Romero &
Gil, 2008).
At the end of the last decade, two great tech-
nological revolutions occurred that would end
the hegemony of the text on the Social Web. On
the one hand, the penetration of the mobile Inter-
net radically increased, bringing connectivity to a
device equipped with a photographic camera. In
parallel, Facebook reaches the “Tipping Point”,
thanks in turn to a large extent to the develop-
ment of the Mobile Internet. Facebook is the rst
massive social network that facilitates and pro-
motes communication between users based on
images (mostly photos). A few years later, Twitter,
which was born as a social network oriented to
written communication, would migrate towards
audiovisual communication. Later on, Instagram
emerged, giving the text an almost residual role,
purely related to hashtags for visibility, topic
search and connections (IAB Spain, 2018).
While these transformations were taking
place, the “Social Listening” industry and moni-
toring tools stayed anchored in the “textual” dy-
1. Background: From social
listening to e-seeing
The interest of the market research industry for
what consumers share in digital media begins
with the launch of the so-called Web 2.0, which
put in the hands of almost any user the possibili-
ty of expressing themselves through these plat-
forms without the technical and economical bar-
riers that hindered the democratization of this
practice. Suddenly, millions of users began to
share their opinions and experiences related to
consumption and brands, which was quickly
perceived by the industry as an opportunity to
complete consumer knowledge obtained through
traditional sources (surveys, focus group, con-
sumer panels, etc.). Rather than sampled con-
sumers, for the rst time it was possible to listen
to the true, unltered consumer opinion on a real
time basis (Celaya, 2008).
Almost at the same time, a parallel industry,
that of “Social Listening”, was also developed,
led by small technology companies, which put
on the market tools that allowed the collection
and analysis of what was published on the Social
Web from text-based queries. After setting one
or more keywords, these tools connect to the da-
tabases of social networks, via API, extract the
contents in which there is a match with those
keywords, store them and analyze them, both
quantitatively and qualitatively. In short, these
are tools whose main value for the consumer re-
search industry lies in analyzing “what is writ-
ten”, the word.
The development of commercial Social Lis-
tening tools coincides with a period (mid-last
decade) in which the Social Web was dominat-
ed by platforms in which the text had a clearly
greater weight than the image, as was the case of
1 Thanks to: Nethodology & Microsoft
Evolving Social Insights from E-Listening to E-Seeing: The L’Oréal Case · págs. 8 a 29
namics of the origins of the Social Web. This
meant not only a loss of lots of information that
consumers were sharing in social networks, but
also a real challenge for sectors, such as beauty
or fashion, in which the user is constantly com-
municating their tastes and preferences in these
consumer categories, even when the message
they want to convey has nothing to do with it. In
practically any photo uploaded to Instagram in
which a person appears, there will almost cer-
tainly be an outt, a hairstyle and a facial look
(even if it is a completely clean face). Suddenly,
missing out your consumers’ communications
in channels that were so disruptive had become
a risk for brands. If you were not able to be in
touch with your consumers in a moment when
they are broadly sharing most of their private life
publicly, you could be perceived as an out of
date brand, losing the connection with your au-
dience. (Borgatti,Everett &Johnson, 2018).
By the late 2000’s, some technological revolu-
tions disrupted the social networks universe.
Firstly, smartphones were massively adopted,
giving online access to users equipped with in-
creasingly better cameras. In parallel, Facebook
drastically switched its philosophy to a mobile,
more image-based network; as an example, to
illustrate this strategy mindset, they even had
‘mobile only days in the Facebook HQ, where all
laptops were shut down and it was only possible
to work via mobile-. Later on, Twitter enabled
better visual content on their feeds. Finally, Ins-
tagram disrupted the market with a pure im-
age-based concept, with very little importance
given to text. Due to high user engagement rate,
Instagram also a valuable social media market-
ing tool. The most important consumer sociali-
zation variables are with an emphasis on peer
communications, brand related factors, and the
use of the social network for eWOM among the
users of Instagram. When consumers follow
brands in a social network, the brands may cre-
ate their desirable attitudes and build loyalty in
consumers. In addition, engagement through
social media platforms can play a very important
role in building brand relationship quality. By
studying how users imitate the behaviour of oth-
er users and their attitudes towards brands, they
can distinguish the users’ perception of relation-
ship quality, their use of social networks, the
number of brands that they follow, and the indi-
viduals who participate in brand-related eWOM.
This distinction can help marketers discover the
most likely perspective of brand management.
(Delafrooz, Rahmati & Abdi, 2019).
Facebook quickly perceived this emerging
trend and purchased the booming app for the
sum of $ 1Bn in April 2012. Considered a rip off
by then, now Facebook’s stock is worth four
times its 2012 average price —and comple-
menting its mature founding social network
with a fresh, younger and wildly image based
platform has denitely played a big role into it.
Not only image is trending, but it is also de-
manded by youngsters— the most intensive users
of social networks (Borgatti et al., 2018).
Furthermore, in the following years, all dis-
ruptive approaches in the social network market
—such as Snapchat stories, which were later
replicated by Instagram— were purely visual.
The latest trend is Tik Tok, a 100 % video-based
network based on short interactions from three
to 60 seconds. Consumers happily embraced
this shift and massively adopted the image-based
communication, therefore limiting the room for
precise text analytics. In a context where an im-
age tells a thousand words, text is only used to
mention, link, or cheer up, rather than provid-
ing an insightful context about a consumer’s feel-
ings or likes.
aDResearch ESIC
Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29
Lots of visual information with key insights
was being missed due to social listening limita-
tions. In this situation, moving from text-based
social listening to image-based, or e-seeing, was
not only a logical next step, but also an absolute
need to continue to be in touch with consumers.
As the world industry leader, it is a must for
L’Oréal to understand how people connect, en-
gage, share and enjoy their beauty across ages,
genders and likes, helping marketers to match
the right audiences, in the right moment, with
the right content, through the right channels
(Borgattiet al., 2018).
The very name of the technique— Social Lis-
tening— already hints that the focus is on the
text; an example of a typical output can be seen
on Figure 1. You “listen” to what is “said”, what is
“written”. As it has been exposed, this approach
leaves out something tat in consumer research is
essential: “what is done”, which, transferred to
the context of social networks, would be “what
is shown”. “We listen”, but we do not “see”
(Ramos et al., 2020).
In conventional qualitative research (through
ethnography, focus group, in-depth interviews,
etc.) the advantages of image analysis are well
known. They are used, for example, to allow
consumers to express ideas that are repressed
(consciously or unconsciously) or that are so
complex that people nd it difcult to verbalize
them. They are also used for the user to project
(as in the collage technique). And, of course,
they play a fundamental role in ethnographic re-
search by providing context information that
hardly appears when other forced context tech-
niques are used, such as focus groups (Malhotra,
1996; Silverman, 2016).
These same advantages are fully extrapolated
to research on social networks. There are mil-
lions of users who have stopped talking to start
showing. It may be asked then why even today
the analysis of images published on social net-
works continues to be marginal. It is necessary
to adopt new tools and develop methodologies
that incorporate this source to enrich analyses
and obtain better insights. In short, move from
Social Listening to e-watching.
Now, more than ever before, ethnographers are
using visual and digital images and technologies to
research and represent the cultures, lives and ex-
periences of other people. Theoretical and techno-
logical innovations have made the visual both ac-
ceptable and accessible to anthropologists and this
has created a contemporary context where new
ethnographic media, methodologies and practices
are emerging (Alfonso, Kurti & Pink, 2004).
Articial intelligence, a great ally. Internet
monitoring tools have begun to take some timid
Figure 1. Classic text query results and topic analysis
Evolving Social Insights from E-Listening to E-Seeing: The L’Oréal Case · págs. 8 a 29
steps for tracing images in which an object of
interest for research appears. Several of the most
relevant tools in this market already have func-
tionalities that allow, for example, detecting im-
ages in which a brand logo appears, even if it is
not mentioned in text.
These advances, still clearly insufcient, have
been possible thanks to the technological boost
given to image recognition by Internet big play-
ers, with Google, Amazon or Microsoft leading
the way. Their need to analyze, classify and
moderate the vast amount of images published
every minute on the Internet is behind the
speed with which these technologies, support-
ed by articial intelligence, have been devel-
oped and democratized. And of their own need,
these companies have found a new market
niche that involves making these technologies
available to third parties. Google has Cloud Vi-
sion, a service via API, which allows to extract
information from images, such as a wide catalog
of objects, recognition of logos or faces, and cre-
ate new custom models.
With time, these tools further developed and
enabled a series of relevant insights, such as in-
corporating objective information from the us-
ers’ proles in social networks, or improving the
semantic analyses to understand the consumers’
sentiment towards a product - whether positive,
negative or neutral, as appreciated on example
Figure 2. This was particularly relevant for un-
derstanding customers’ proles, to manage spe-
cic crises or to analyse the brand engagement
through time, although again, purely based on
text interactions (Alfonso et al., 2004).
As above mentioned, the limitations of these
tools are considerable, while the possibilities
that e-seeing offers are huge. And not only in
terms of insight, but also on scalability. A lan-
guage processing tool faces constraints when
switching to a different language (or even be-
tween different language regions, as Spain or
Latin America could be), and needs to be thor-
oughly reviewed. On the other hand, image pro-
cessing has the enormous power to be automat-
ically escalated to any other country with very
minor tweaks. If you can train a tool to detect
and lter a specic element in an image, that el-
ement can be traced in any picture taken around
the world; now imagine the potential that this
can have applied to a global scale. In the case of
L’Oréal, it can be possible to leverage local devel-
opments, and a small initiative can completely
change the way the entire organizations relates
with their customers in a relatively short period
of time – denitely shorter than the usual tim-
ings needed for long-scale organizational chang-
es (Kaneshige, 2015).
Figure 2. Gender, interest and sentiment analysis
aDResearch ESIC
Nº 23 Vol 23 · Segundo semestre, julio-diciembre 2020  págs. 8 a 29
L’Oréal itself is an example on how im-
age-based IA can be easily escalated, not only
geographically, but also in the way that a simi-
lar algorithm (identifying objects in an image)
can be utilized for different functions. The fol-
lowing tools were recently deployed within the
L’Oréal insights community and have helped
the teams since:
• Deep Vision (Figures 3-4): Image recogni-
tion and linkage to other similar ones on
Instagram, enabling the classication of
over 50,000 images into different clusters
of similarity and tracing of its brand. Visu-
alization shown below.
This tool enables L’Oréal teams to monitor the
key trends and competitor activity, as well as ad-
Figure 3. Deep Vision make-up landing screen
Figure 4. Deep Vision zoom on eye make-up and NYX professional make-up
Evolving Social Insights from E-Listening to E-Seeing: The L’Oréal Case · págs. 8 a 29
just campaign investment weights based on their
axis strategy. It is also used to visualise examples
around given trends.
• Golden Eye (Figure 5): the same algorithm,
but focused on quick and sharp image se-
lection for marketing teams. This tool high-
lights the most relevant elements of a pic-
ture and contrasts them with Microsoft’s
and MIT’s database of neuronal studies:
over 70,000 cases in total to match with
our algorithm, to predict image salience
and memorability. This way, marketing
teams can quickly make choices between
multiple ambassadors, creativities, product
images and image compositions, with a sol-
id criteria behind and with the help of AI.
2. The approach of L’Oréal Insights:
Partnering with AI
When rst facing the challenge of making this
relevant shift, the L’Oréal Consumer & Market
Intelligence team searched for inspiration among
well known traditional techniques. Additionally,
Figure 5. GoldenEye interface and results
in the same way images were a great support for
conventional qualitative techniques, the goal
was to develop similar capabilities, but with a
more ambitious scope. In traditional qualitative
research, showing images directly to consumers
is often a good approach for gathering rst-
hand, direct reactions, but it is also useful to use
projection techniques to dig deeper into con-
sumers’ feelings, and understand the underlying
perceptions that matter the most - the true in-
sight. Translating this philosophy into e-seeing,
the goal was to not only perceive the evident,
but also spot the trend, identify the tribe and
separate trend-setters from the followers. At the
opposite end with a pure tech approach, AI was
taking its rst steps into the social listening in-
dustry. The rst tools enabled logo identication
within sets of images, or basic information: gen-
der, age, etc. Gradually, big players entered the
market, such as Google with Cloud Vision, or
Microsoft with Computer Vision. These were
more powerful tools, enabling a deeper analysis
for the Consumer & Market Intelligence teams
(Mulfari, 2016; Del Sole, 2017).