Facial expression tracking during augmented and virtual reality sessions
An example method for estimating an emotion based upon a facial expression of a user can include: receiving one or more captured facial expressions from the user at a visual computing device; comparing the one or more captured facial expressions to one or more known facial expressions; and assigning an emotion to the plurality of captured facial expressions based upon the comparing.
1. A visual computing device that can be worn by a user, the visual computing device comprising:
a processing unit;
system memory;
a display unit on which one or more virtual images can be projected; and
a plurality of cameras, the plurality of cameras being oriented in an inwardly-facing direction towards a face of the user to capture facial reactions of the user;
wherein the system memory encodes instructions that, when executed by the processing unit, cause the visual computing device to:
display, by the display unit, an image to the user, the image being of an item the user may be interested in purchasing;
capture, using one or more of the plurality of cameras, a facial expression of the user as a result of the user viewing the image on the display unit, the facial expression comprising a reaction of the user to the image;
send the captured facial expression to an electronic computing device;
receive, from the electronic computing device, a message identifying an emotion of the user corresponding to the facial expression, the emotion of the user being identified via a comparison of the captured facial expression with a previous facial expression of the user obtained during a training period; and
when the emotion corresponding to the captured facial expression is a negative emotion, display, by the display unit, an image of a different item the user may be interested in purchasing that may create a positive emotional response for the user, wherein the image of the item includes a street scene and the item is a home on the street scene, and wherein the image of the different item includes a different street scene in a different geographical location than the image of the item.
2. The visual computing device of claim 1 , wherein each of the plurality of cameras is located within a field of view of the user.
3. The visual computing device of claim 1 , wherein the visual computing device is a virtual reality device.
4. The visual computing device of claim 1 , wherein the visual computing device is an augmented reality device.
5. The visual computing device of claim 4 , wherein the augmented reality device further includes one or more outwardly-facing cameras which are oriented away from the face of the user.
6. The visual computing device of claim 1 , wherein the image of the different item the user may be interested in purchasing includes a more expensive home than in the image of the item.
7. The visual computing device of claim 1 , wherein the image of the different item the user may be interested in purchasing includes a less expensive home than in the image of the item.
8. The visual computing device of claim 1 , wherein the image of the item is an automobile, and the image of the different item is a more expensive or less expensive automobile.
9. A method implemented on a virtual reality device, the method comprising:
displaying, by the virtual reality device, a first image to a user, the first image being a street scene comprising a plurality of homes;
orienting one or more inwardly-facing cameras of the virtual reality device towards a face of the user;
capturing a facial expression of the user, using the one or more inwardly-facing cameras, as a result of the user viewing the first image on the virtual reality device, the facial expression comprising a reaction of the user to the first image;
sending the captured facial expression to an electronic computing device;
receiving, from the electronic computing device, a message identifying a user emotion associated with the facial expression, the user emotion associated with the facial expression being identified via a comparison of the captured facial expression with a previous facial expression of the user obtained during a training period;
determining whether the user emotion is negative; and
when the user emotion is negative, displaying a second image to the user, the second image being a street scene with different homes than in the first image, and the street scene in the second image being in a different geographical location than street scene in the first image.
10. The method of claim 9 , further comprising selecting the second image to produce a positive emotion by the user.
11. The method of claim 9 , further comprising:
displaying one or more possible actions to be taken by the user to improve the user emotion; and
capturing a second facial expression relating to the one or more possible actions.
12. The method of claim 11 , further comprising determining the user emotion associated with the one or more possible actions based upon the second facial expression.
13. The method of claim 9 , wherein the second image contains smaller, less expensive homes than the first image.
14. The method of claim 9 , wherein the second image contains more expensive homes than the first image.
15. A visual computing device that can be worn by a user, the visual computing device comprising:
a processing unit;
system memory;
a display unit on which one or more virtual images can be projected; and
a plurality of cameras, the plurality of cameras being oriented in an inwardly-facing direction towards a face of the user to capture facial reactions of the user;
wherein the system memory encodes instructions that, when executed by the processing unit, cause the visual computing device to:
display, by the display unit, an image to the user, the image being of an item the user may be interested in purchasing;
capture, using one or more of the plurality of cameras, a facial expression of the user as a result of the user viewing the image on the display unit, the facial expression comprising a reaction of the user to the image;
send the captured facial expression to an electronic computing device;
receive, from the electronic computing device, a message identifying an emotion of the user corresponding to the facial expression, the emotion of the user being identified via a comparison of the captured facial expression with a previous facial expression of the user obtained during a training period; and
when the emotion corresponding to the captured facial expression is a negative emotion, display, by the display unit, an image of a different item the user may be interested in purchasing that may create a positive emotional response for the user, wherein the image of the item is a vacation scene, and the image of the different item is a vacation scene in a different geographical location than for the image of the item.