FACE MODEL CAPTURE BY A WEARABLE DEVICE

1 . A system for generating a three-dimensional (3D) model of a face of a user, the system comprising:

a head-mounted device (HMD) associated with an inward-facing camera, wherein the inward-facing camera is configured to image at least a portion of the face of the user while the user is wearing the HMD;

an inertial measurement unit (IMU) associated with the HMD and configured to detect movements of the HMD; and

a hardware processor programmed to:

detect a trigger to initiate imaging of a face of the user, wherein the trigger comprises a movement detected by the IMU involving putting the HMD onto a head of the user or taking the HMD off of the head of the user;

activate, in response to detecting the trigger, the inward-facing camera to acquire images; and

detect a stopping condition for stopping the imaging based on data acquired from at least one of the IMU and the inward-facing camera.

2 . The system of claim 1 , wherein the hardware processor is also programmed to:

analyze the images acquired by the inward-facing camera with a stereo vision algorithm; and

fuse the images to generate a face model of the user's face based at least partly on an output of the stereo vision algorithm.

3 . The system of claim 2 , wherein the stereo vision algorithm comprises at least one of: a block-matching algorithm, a semi-global matching algorithm, a semi-global block-matching algorithm, a disparity map, a depth map, or a neural network algorithm.

4 . The system of claim 2 , further comprising a second camera, and wherein the inward-facing camera and the second camera have an overlapping field of view.

5 . The system of claim 4 , wherein the images comprise a plurality of pairs of images, wherein each pair of images comprises a first image acquired by the inward-facing camera and a second image acquired by the second camera.

6 . The system of claim 5 , wherein a pair of images is analyzed together with the stereo vision algorithm.

7 . The system of claim 5 , wherein the output of the stereo vision algorithm comprises depth assignments to pixels in the plurality of pairs of images.

8 . The system of claim 5 , wherein the user's face is represented by a plurality of point clouds based on the analysis of the images acquired by the inward-facing camera and the second camera, and wherein to fuse the images to generate a face model, the hardware processor is programmed to:

fit the plurality of clouds to one another;

reject outliers in the plurality of clouds; and

smooth a surface of the face model by at least one of clustering or averaging.

9 . The system of claim 8 , wherein to fit the plurality of clouds, the hardware processor is programmed to apply Iterative Closest Point algorithm to the plurality of clouds.

10 . The system of claim 2 , wherein to analyze the images, the hardware processor is programmed to at least:

identify keypoints in the images using a keypoints detector and descriptor algorithm; or

identify facial features from the images and describe the identified facial features with points in a 3D space.

11 . The system of claim 10 , wherein to fuse the images, the hardware processor is programmed to combine the keypoints or facial features using a bundle adjustment algorithm.

12 . The system of claim 1 , wherein to detect the trigger, the hardware processor is programmed to:

determine an acceleration of the HMD;

compare the acceleration of the HMD with a threshold acceleration; and

detect the trigger in response to a comparison that the acceleration exceeds the threshold acceleration.

13 . The system of claim 1 , wherein the stopping condition is detected when a distance between the HMD and the head of the user passes a threshold distance.

14 . The system of claim 1 , wherein the hardware processor is further programmed to:

determine a texture map based on the images; and

apply the texture map to the face model.

15 . The system of claim 1 , wherein the hardware processor is further programmed to pass the face model to a wearable device.

16 . A method for generating a three-dimensional (3D) model of a face of a user, the method comprising:

receiving a request for generating a face model of a user;

accessing images of the user's head acquired by an inward-facing camera of a wearable device;

identifying a plurality of pairs of images from the accessed images;

analyzing the images by applying a stereo vision algorithm to the plurality of pairs of images; and

fusing outputs obtained from said analyzing step to create a face model,

wherein the images are acquired as the wearable is being put on or taken off from the user.

17 . The method of claim 16 , wherein the outputs comprise a depth map associated with the user's face, which contains information relating to distances between the face and the wearable device.

18 . The method of claim 16 , wherein the wearable device comprises the inward-facing camera and a second camera, and a pair of images comprises a first image and a second image that are acquired at substantially the same time by the inward-facing camera and the second camera respectively.

19 . The method of claim 16 , wherein analyzing the images comprises converting the plurality of pairs of images into point clouds.

20 . The method of claim 19 , wherein fusing the outputs comprises combining the point clouds using an iterative closest point algorithm.