Facial expressions from eye-tracking cameras

1. A wearable display system comprising:

an inward-facing imaging system configured to capture an image of a periocular region of a wearer of the wearable display system;

non-transitory memory configured to store the image of the periocular region; and

a hardware processor in communication with the non-transitory memory, the hardware processor programmed to:

access the image of the periocular region of the wearer;

determine, based at least partly on the image, one or more periocular features of the wearer;

access a mapping table indicating associations between periocular features and lower face parameters, wherein the lower face parameters indicate different deformations of the lower face of the wearer;

determine, based on the mapping table, one or more lower face parameters associated with the one or more periocular features; and

generate a representation of a lower face of the wearer based on the determined lower face parameters so that the representation of the lower face indicates a facial expression associated with the determined periocular features.

2. The wearable display system of claim 1 , wherein the periocular features include extensions and compressions of periocular muscles, wrinkles, blemishes, or patches in the periocular region, eyelids, or eyebrows.

3. The wearable display system of claim 1 , wherein the hardware processor is further programmed to:

determine, based at least partly on the image, one or more periocular region parameters including optical flow of a periocular feature or measurement of a raise or a lowering of an eyebrow.

4. The wearable display system of claim 3 , wherein the one or more periocular region parameters include a vertical eye center (VEC) position.

5. The wearable display system of claim 1 , wherein the inward-facing imaging system comprises an infrared light emitting diode (IR LED) or a photo-reflective sensor configured to detect a light intensity of the periocular region.

6. The wearable display system of claim 1 , wherein the hardware processor is further programmed to:

measure a light intensity of a patch in the periocular region; and

compare the light intensity of the patch against a threshold to determine a distance of the periocular region underlying the patch from the inward-facing imaging system.

7. The wearable display system of claim 1 , wherein a lower face conformation is cross-checked for full face consistency based on an eyebrow position.

8. The wearable display system of claim 1 , further comprising an audio sensor, wherein a lower face conformation is checked for consistency against phonemes recorded from the audio sensor.

9. The wearable display system of claim 1 , wherein a fit level of the wearable display system on the wearer is determined based on eye positions.

10. The wearable display system of claim 9 , wherein the eye positions are determined from pupil-glint tracking.

11. The wearable display system of claim 1 , further comprising an electrooculography (EOG) sensor, wherein the wearable display system is configured to determine eye positions from readings of EOG sensor.

12. The wearable display system of claim 1 , wherein the wearable display system is configured to obtain calibration data specific to the wearer.

13. The wearable display system of claim 12 , wherein the wearable display system is configured to obtain a second image of a region of the face of the wearer that is not imaged by the inward-facing imaging system, and wherein the calibration data comprises a head shape or a mouth shape.

14. The wearable display system of claim 1 , wherein the wearable display system is configured to temporally check the representation of the lower face of the wearer against past lower face representations generated from previously captured periocular region images.

15. A method comprising:

under control of a wearable display system comprising an inward-facing imaging system configured to capture an image of a periocular region of a wearer of the wearable display, non-transitory memory configured to store the image of the periocular region; and a hardware processor in communication with the non-transitory memory:

accessing the image of the periocular region of the wearer;

determining, based at least partly on the image, one or more periocular features of the wearer;

accessing a mapping table indicating associations between periocular features and lower face parameters, wherein the lower face parameters indicate different deformations of the lower face of the wearer;

determining, based on the mapping table, one or more lower face parameters associated with the one or more periocular features; and

generating a representation of a lower face of the wearer based on the determined lower face parameters so that the representation of the lower face indicates a facial expression associated with the determined periocular features.

16. A non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by a hardware computer processor to cause a computing system to perform operations comprising:

accessing an image of a periocular region of a wearer of the computing system;

determining, based at least partly on the image, one or more periocular features of the wearer;

accessing a mapping table indicating associations between periocular features and lower face parameters, wherein the lower face parameters indicate different deformations of the lower face of the wearer;

determining, based on the mapping table, one or more lower face parameters associated with the one or more periocular features; and

generating a representation of a lower face of the wearer based on the determined lower face parameters so that the representation of the lower face indicates a facial expression associated with the determined periocular features.