DETERMINING MICRO-EXPRESSIONS OF A USER
In some examples, while a user is navigating a site using a browser, a computing device may receive one or more images from a camera and may monitor input data from input devices (e.g., mouse, keyboard). After a particular event occurs (e.g., a navigation event such as selecting a link, selecting a tab, scrolling up or down, or the like), the computing device may analyze the images captured after the event using a machine learning algorithm to determine a micro-expression of the user. The micro-expression may be classified as a particular sentiment of a plurality of sentiments, associated with the event, and sent to a server. The server or the computing device may instruct the browser to modify, based on the sentiment, a portion of the site. The modification may include displaying a user interface to enable the user to communicate with a representative associated with the site.
1 . A method comprising:
receiving, by one or more processors of a computing device, input data from one or more input devices being used to navigate a site displayed by a browser;
determining, by the one or more processors and based on the input data and the site, that an event occurred;
receiving, by the one or more processors, one or more images from a camera that is connected to the computing device;
performing an analysis, by the one or more processors, of at least one image of the one or more images;
determining, by the one or more processors and based on the analysis, that the at least one image includes a micro-expression of a user that is using the computing device;
determining, by the one or more processors, a sentiment corresponding to the micro-expression;
associating, by the one or more processors, the sentiment with the event; and
sending, by the one or more processors, the sentiment and the event to a server.
2 . The method of claim 1 , further comprising:
determining that the sentiment is not a happy sentiment; and
sending one or more instructions to the browser, wherein the one or more instructions cause the browser to modify at least a portion of the site being displayed.
3 . The method of claim 2 , wherein modifying at least the portion of the site being displayed comprises at least one of:
increasing a font size of at least the portion of the site; or
increasing an image size of an image included in the portion of the site.
4 . The method of claim 2 , wherein modifying at least the portion of the site being displayed comprises:
displaying a contact user interface to enable the user to either contact a representative or be contacted by a representative, wherein the contact comprises one or more of a call, a chat, or an email.
5 . The method of claim 2 , further comprising:
receiving one or more additional images from the camera;
performing an additional analysis of the one or more additional images;
determining based on the additional analysis, that the one or more additional images include a second micro-expression;
determining a second sentiment corresponding to the second micro-expression; and
determining that the second sentiment is the happy sentiment.
6 . The method of claim 1 , wherein the event comprises one of:
selecting a tab to navigate to a particular portion of the site;
selecting a hyperlink to navigate to the particular portion of the site;
selecting a menu item to navigate to the particular portion of the site; or
scrolling up or down a page that is being displayed on the site to access the particular portion of the site.
7 . The method of claim 1 , wherein the sentiment comprises one of:
a neutral sentiment, a surprise sentiment, a fear sentiment, a disgust sentiment, an angry sentiment, a happy sentiment, a sad sentiment, or a contempt sentiment.
8 . A computing device comprising:
one or more processors; and
one or more non-transitory computer-readable storage media to store instructions executable by the one or more processors to perform operations comprising:
receiving input data from one or more input devices being used to navigate a site displayed by a browser;
determining, based on the input data and the site, that an event occurred;
receiving one or more images from a camera that is connected to the computing device;
performing an analysis of at least one image of the one or more images;
determining, based on the analysis, that the at least one image includes a micro-expression of a user that is using the computing device;
determining a sentiment corresponding to the micro-expression;
associating the sentiment with the event; and
sending the sentiment and the event to a server.
9 . The computing device of claim 8 , wherein the operations further comprise:
determining that the sentiment is not a happy sentiment; and
sending one or more instructions to the browser, wherein the one or more instructions cause the browser to modify at least a portion of the site being displayed.
10 . The computing device of claim 9 , wherein modifying at least the portion of the site being displayed comprises at least one of:
increasing a font size of at least the portion of the site; or
increasing an image size of an image included in the portion of the site.
11 . The computing device of claim 9 , wherein modifying at least the portion of the site being displayed comprises:
displaying a contact user interface to enable the user to either contact a representative or be contacted by a representative, wherein the contact comprises one or more of a call, a chat, or an email.
12 . The computing device of claim 9 , further comprising:
receiving one or more additional images from the camera;
performing an additional analysis of the one or more additional images;
determining based on the additional analysis, that the one or more additional images include a second micro-expression;
determining a second sentiment corresponding to the second micro-expression; and
determining that the second sentiment is the happy sentiment.
13 . The computing device of claim 8 , wherein the event comprises one of:
selecting a tab to navigate to a particular portion of the site;
selecting a hyperlink to navigate to the particular portion of the site;
selecting a menu item to navigate to the particular portion of the site; or
scrolling up or down a page that is being displayed on the site to access the particular portion of the site.
14 . The computing device of claim 8 , wherein, before receiving the one or more images from the camera, the operations further comprise:
determining that the user has not provided permission to receive the one or more images from the camera;
displaying a message requesting the permission to receive the one or more images from the camera; and
displaying an incentive to provide the permission.
15 . One or more non-transitory computer readable media storing instructions executable by an embedded controller to perform operations comprising:
receiving input data from one or more input devices being used to navigate a site displayed by a browser;
determining, based on the input data and the site, that an event occurred;
receiving one or more images from a camera that is connected to the computing device;
performing an analysis of at least one image of the one or more images;
determining, based on the analysis, that the at least one image includes a micro-expression of a user that is using the computing device;
determining a sentiment corresponding to the micro-expression;
associating the sentiment with the event; and
sending the sentiment and the event to a server.
16 . The one or more non-transitory computer readable media of claim 15 , wherein the operations further comprise:
determining that the sentiment is not a happy sentiment; and
sending one or more instructions to the browser, wherein the one or more instructions cause the browser to modify at least a portion of the site being displayed.
17 . The one or more non-transitory computer readable media of claim 16 , wherein modifying at least the portion of the site being displayed comprises at least one of:
increasing a font size of at least the portion of the site; or
increasing an image size of an image included in the portion of the site.
18 . The one or more non-transitory computer readable media of claim 16 , wherein modifying at least the portion of the site being displayed comprises:
displaying a contact user interface to enable the user to either contact a representative or be contacted by a representative, wherein the contact comprises one or more of a call, a chat, or an email.
19 . The one or more non-transitory computer readable media of claim 16 , receiving one or more additional images from the camera;
performing an additional analysis of the one or more additional images;
determining based on the additional analysis, that the one or more additional images include a second micro-expression;
determining a second sentiment corresponding to the second micro-expression; and
determining that the second sentiment is the happy sentiment.
20 . The one or more non-transitory computer readable media of claim 15 , wherein, before receiving the one or more images from the camera, the operations further comprise:
determining that the user has not provided permission to receive the one or more images from the camera;
displaying a message requesting the permission to receive the one or more images from the camera; and
displaying an incentive to provide the permission.