Face model capture by a wearable device
Systems and methods for generating a face model for a user of a head-mounted device are disclosed. The head-mounted device can include one or more eye cameras configured to image the face of the user while the user is putting the device on or taking the device off. The images obtained by the eye cameras may be analyzed using a stereoscopic vision technique, a monocular vision technique, or a combination, to generate a face model for the user.
1. A system for generating a three-dimensional (3D) model of a face of a user, the system comprising:
a head-mounted display (HMD) configured to present virtual content to a user;
an inward-facing imaging system comprising a first eye camera and a second eye camera, wherein the first and second eye cameras are configured to image left and right eyes of the user, respectively, while the user is wearing the HMD; and
a hardware processor programmed to:
acquire images of the face of the user using the first and second eye cameras;
based at least on sensor data from one or more inertial sensor of the HMD, detect movement of the HMD indicative of movement towards the head of the user;
in response to said detected movement, for pairs of acquired images each including a first image from the first eye camera and a second eye image from the second eye camera:
determine, based on one or more of the acquired images, a distance between the head of the user and the HMD; and
in response to determining that the distance is within a range of distances,
tag the pair of acquired images as usable in generating a 3D model of the face of the user;
identify one or more tagged pairs of acquired images;
for each of the one or more tagged pairs of acquired images, analyze the acquired images with a stereo vision algorithm; and
based at least on an output of the stereo vision algorithm, fuse the one or more tagged pairs of acquired images to generate a 3D model of the face of the user.
2. The system of claim 1 , wherein the first eye camera and the second eye camera have an overlapping field of view.
3. The system of claim 2 , wherein the overlapping field of view of the first eye camera and the second eye camera defines an imaging region, and the certain range of distances comprise distances within the imaging region.
4. The system of claim 1 , wherein the hardware processor is programmed to detect, from at least one sensor associated with the HMD, a worn position of the HMD on the user.
5. The system of claim 4 , wherein the at least one sensor comprises a proximity sensor.
6. The system of claim 5 , wherein the proximity sensor comprises a capacitive proximity sensor.
7. The system of claim 5 , wherein data from the proximity sensor comprises a distance between the user's head and the proximity sensor.
8. The system of claim 4 , wherein the at least one sensor comprises a pressure sensor.
9. The system of claim 8 , wherein data from the pressure sensor comprises a pressure measurement.
10. The system of claim 8 , wherein the pressure sensor is disposed on a temple of the HMD.
11. The system of claim 8 , wherein the pressure sensor is disposed on a nose pad of the HMD.
12. The system of claim 4 , wherein the at least one sensor comprises an inertial measurement unit (IMU).
13. The system of claim 12 , wherein data from the IMU comprises an acceleration measurement of the HMD.
14. The system of claim 4 , wherein the at least one sensor comprises the first eye camera or the second eye camera.
15. The system of claim 14 , wherein data from the first eye camera or the second eye camera comprises an image of the user's face.
16. The system of claim 1 , wherein to determine the distance between the head of the user and the HDM, the hardware processor is programmed to:
analyze the pair of acquired images to estimate a size of the face of the user in an imaging plane associated with the pair of acquired images; and
determine whether the size of the face of the user is less than a threshold size, wherein the threshold size is associated with the distance being outside of the certain range of distances.
17. A method for generating a three-dimensional (3D) model of a face of a user, the method comprising:
receiving a request for generating a 3D face model of a user;
based at least on sensor data from one or more inertial sensor of a head-mounted display (HMD), detect movement of the HMD indicative of movement towards the head of the user;
in response to said detected movement,
accessing images acquired by an inward-facing imaging system of a wearable device, wherein the inward-facing imaging system comprises a first eye camera and a second eye camera;
tagging at least one image of the accessed images in response to determining that the user's head position is within a certain range of distances,
wherein the user's head position is based on a size of the face of the user in the at least one image of the accessed images;
identifying a plurality of pairs of images from the tagged images;
analyzing the tagged images by applying a stereo vision algorithm to the plurality of pairs of images; and
fusing outputs obtained from said analyzing to create the 3D model of the face of the user.
18. The method of claim 17 , wherein the outputs comprise a depth map associated with the face of the user, which contains information relating to distances between the face and the wearable device.
19. The method of claim 17 , wherein a pair of images comprises a first image and a second image that are acquired substantially concurrently by the first eye camera and the second eye camera respectively.
20. The method of claim 17 , wherein analyzing the images comprise converting the plurality of pairs of images into point clouds.
21. The method of claim 20 , wherein fusing the outputs comprises combining the point clouds using an iterative closest point algorithm.
22. The system of claim 17 , wherein the certain range of distances comprise distances greater than 175 mm.
23. The system of claim 17 , wherein the certain range of distances comprise distances greater than a distance where images from the first eye camera or the second eye camera are out of focus.
24. The method of claim 17 , wherein a field of view of the first eye camera and a field of view of the second eye camera overlap to define an imaging region, and the certain range of distances comprise distances within the imaging region.
25. The method of claim 24 , wherein the certain range of distances comprise distances greater than 175 mm.
26. The method of claim 25 , wherein the certain range of distances comprise distances greater than a distance where images from the first eye camera or the second eye camera are out of focus.