Constructing a user's face model using particle filters

1. A system, comprising:

a processor configured to:

receive a plurality of images of a user's face at various orientations;

determine a first new plurality of sets of extrinsic camera information particles corresponding to respective ones of the plurality of images of the face, including by applying an extrinsic camera particle filter to determine weights corresponding to previously obtained sets of extrinsic camera information particles corresponding to a first image using a first selected face model particle;

select a subset of the first new plurality of sets of extrinsic camera information particles;

determine a first new plurality of face model particles corresponding to the plurality of images of the face, including by inputting the selected subset of the first new plurality of sets of extrinsic camera information particles into a face model particle filter to determine weights corresponding to a previously obtained plurality of face model particles corresponding to the plurality of images of the face;

select a second face model particle included in the first new plurality of face model particles;

determine a second new plurality of sets of extrinsic camera information particles corresponding to respective ones of the plurality of images of the face based at least in part on the second selected face model particle; and

produce a 3D face model based at least in part on the first new plurality of face model particles and the second new plurality of sets of extrinsic camera information; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. The system of claim 1 , wherein the first selected face model particle comprises a model of a generic face.

3. The system of claim 1 , wherein the first selected face model particle comprises a historical model of a historical user's face.

4. The system of claim 1 , wherein the processor is further configured to select an initial plurality of sets of extrinsic camera information particles corresponding to respective ones of the plurality of images of the face from a distribution of historical sets of extrinsic camera information.

5. The system of claim 1 , wherein the processor is further configured to select an initial plurality of face model particles from a distribution of historical models of historical users' faces.

6. The system of claim 1 , wherein the processor is further configured to select an initial plurality of face model particles from a distribution of generic faces.

7. The system of claim 1 , wherein the application of the extrinsic camera particle filter includes:

performing resampling on the previously obtained sets of extrinsic camera information particles corresponding to the first image based at least in part on the weights to generate resampled previously obtained sets of extrinsic camera information particles corresponding to the first image; and

outputting new sets of extrinsic camera information particles corresponding to the first image based at least in part on the resampled previously obtained sets of extrinsic camera information particles corresponding to the first image.

8. The system of claim 7 , wherein to generate the new sets of extrinsic camera information particles corresponding to the first image based at least in part on the resampled previously obtained sets of extrinsic camera information particles corresponding to the first image comprises to:

determine two corresponding resampled previously obtained sets of extrinsic camera information particles from two neighboring images, wherein the two neighboring images include the first image;

determine a first previously obtained set of extrinsic camera information particle of the two corresponding previously obtained sets of extrinsic camera information particles as a donor particle and a second previously obtained set of extrinsic camera information particle of the two corresponding previously obtained sets of extrinsic camera information particles as an acceptor particle; and

modify the acceptor particle based at least in part on the donor particle and an optical flow between the two neighboring images to output a new set of extrinsic camera information particle corresponding to the first image.

9. The system of claim 1 , wherein to determine weights corresponding to previously obtained sets of extrinsic camera information particles corresponding to the first image using the first selected face model particle comprises to:

transform the first selected face model particle using a previously obtained set of extrinsic camera information particle corresponding to the first image;

project the transformed selected face model particle onto a two-dimensional (2D) surface to generate a 2D projection;

generate measurements associated with the 2D projection; and

determine a weight corresponding to the previously obtained set of extrinsic camera information particle corresponding to the first image based at least in part on the measurements associated with the 2D projection.

10. The system of claim 1 , wherein the application of the face model particle filter includes:

performing resampling on the previously obtained plurality of face model particles corresponding to the plurality of images of the face based at least in part on the weights to generate resampled previously obtained plurality of face model particles; and

outputting the first new plurality of face model particles corresponding to the plurality of images of the face based at least in part on the resampled previously obtained plurality of face model particles corresponding to the plurality of images of the face.

11. The system of claim 10 , wherein to output the first new plurality of face model particles corresponding to the plurality of images of the face based at least in part on the resampled previously obtained plurality of face model particles corresponding to the plurality of images of the face comprises to:

select a resampled previously obtained face model particle;

select a face model from a distribution of face models; and

combine the selected resampled face model particle with the face model that is selected from the distribution to output a new face model particle.

12. The system of claim 1 , wherein to determine weights corresponding to face model particles corresponding to the plurality of images of the face using the selected subset of the first new plurality of sets of extrinsic camera information particles comprises to:

transform a previously obtained face model particle using a selected set of extrinsic camera information particle corresponding to a second image;

project the transformed face model onto a two-dimensional (2D) surface to generate a 2D projection;

generate measurements associated with the 2D projection; and

determine a weight corresponding to the previously obtained face model particle based at least in part on the measurements associated with the 2D projection.

13. The system of claim 1 , wherein:

the first selected face model particle includes a 3D model of a face;

the previously obtained set of extrinsic camera information particles includes at least one of: a rotation angle and a translation; and

the determination of weights corresponding to the previously obtained sets of extrinsic camera information particles includes orienting the 3D model based at least in part on at least one of a rotation angle and a translation included in the previously obtained sets of extrinsic camera information particles.

14. A method, comprising:

receiving a plurality of images of a user's face at various orientations;

determining a first new plurality of sets of extrinsic camera information particles corresponding to respective ones of the plurality of images of the face, including by applying an extrinsic camera particle filter to determine weights corresponding to previously obtained sets of extrinsic camera information particles corresponding to a first image using a first selected face model particle;

selecting a subset of the first new plurality of sets of extrinsic camera information particles;

determining a first new plurality of face model particles corresponding to the plurality of images of the face, including by inputting the selected subset of the first new plurality of sets of extrinsic camera information particles into a face model particle filter to determine weights corresponding to a previously obtained plurality of face model particles corresponding to the plurality of images of the face;

selecting a second face model particle included in the first new plurality of face model particles;

determining a second new plurality of sets of extrinsic camera information particles corresponding to respective ones of the plurality of images of the face based at least in part on the second selected face model particle; and

producing a 3D face model based at least in part on the first new plurality of face model particles and the second new plurality of sets of extrinsic camera information.

15. The method of claim 14 , wherein the application of the extrinsic camera particle filter includes:

performing resampling on the previously obtained sets of extrinsic camera information particles corresponding to the first image based at least in part on the weights to generate resampled previously obtained sets of extrinsic camera information particles corresponding to the first image; and

outputting new sets of extrinsic camera information particles corresponding to the first image based at least in part on the resampled previously obtained sets of extrinsic camera information particles corresponding to the first image.

16. The method of claim 14 , wherein determining weights corresponding to previously obtained sets of extrinsic camera information particles corresponding to the first image using the first selected face model particle comprises:

transforming the first selected face model particle using a previously obtained set of extrinsic camera information particle corresponding to the first image;

projecting the transformed selected face model particle onto a two-dimensional (2D) surface to generate a 2D projection;

generating measurements associated with the 2D projection; and

determining a weight corresponding to the previously obtained set of extrinsic camera information particle corresponding to the first image based at least in part on the measurements associated with the 2D projection.

17. The method of claim 15 , wherein generating the new sets of extrinsic camera information particles corresponding to the first image based at least in part on the resampled previously obtained sets of extrinsic camera information particles corresponding to the first image comprises:

determining two corresponding resampled previously obtained sets of extrinsic camera information particles from two neighboring images, wherein the two neighboring images include the first image;

determining a first previously obtained set of extrinsic camera information particle of the two corresponding previously obtained sets of extrinsic camera information particles as a donor particle and a second previously obtained set of extrinsic camera information particle of the two corresponding previously obtained sets of extrinsic camera information particles as an acceptor particle; and

modifying the acceptor particle based at least in part on the donor particle and an optical flow between the two neighboring images to output a new set of extrinsic camera information particle corresponding to the first image.

18. The method of claim 14 , wherein the application of the face model particle filter includes:

performing resampling on the previously obtained plurality of face model particles corresponding to the plurality of images of the face based at least in part on the weights to generate resampled previously obtained plurality of face model particles; and

outputting the first new plurality of face model particles corresponding to the plurality of images of the face based at least in part on the resampled previously obtained plurality of face model particles corresponding to the plurality of images of the face.

19. The method of claim 14 , wherein determining weights corresponding to face model particles corresponding to the plurality of images of the face using the selected subset of the first new plurality of sets of extrinsic camera information particles comprises:

transforming a previously obtained face model particle using a selected set of extrinsic camera information particle corresponding to a second image;

projecting the transformed face model onto a two-dimensional (2D) surface to generate a 2D projection;

generating measurements associated with the 2D projection; and

determining a weight corresponding to the previously obtained face model particle based at least in part on the measurements associated with the 2D projection.

20. The method of claim 18 , wherein outputting the first new plurality of face model particles corresponding to the plurality of images of the face based at least in part on the resampled previously obtained plurality of face model particles corresponding to the plurality of images of the face comprises:

selecting a resampled previously obtained face model particle;

selecting a face model from a distribution of face models; and

combining the selected resampled face model particle with the face model that is selected from the distribution to output a new face model particle.

21. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving a plurality of images of a user's face at various orientations;

determining a first new plurality of sets of extrinsic camera information particles corresponding to respective ones of the plurality of images of the face, including by applying an extrinsic camera particle filter to determine weights corresponding to previously obtained sets of extrinsic camera information particles corresponding to a first image using a first selected face model particle;

selecting a subset of the first new plurality of sets of extrinsic camera information particles;

determining a first new plurality of face model particles corresponding to the plurality of images of the face, including by inputting the selected subset of the first new plurality of sets of extrinsic camera information particles into a face model particle filter to determine weights corresponding to a previously obtained plurality of face model particles corresponding to the plurality of images of the face;

selecting a second face model particle included in the first new plurality of face model particles;

determining a second new plurality of sets of extrinsic camera information particles corresponding to respective ones of the plurality of images of the face based at least in part on the second selected face model particle; and

producing a 3D face model based at least in part on the first new plurality of face model particles and the second new plurality of sets of extrinsic camera information.