Apparatuses and methods for face tracking based on calculated occlusion probabilities

1. A face tracking method, comprising:

detecting a face region based on an input image;

segmenting the face region into a plurality of sub-regions;

calculating occlusion probabilities for the plurality of sub-regions, the occlusion probabilities for the plurality of sub-regions calculated based on probability models for a plurality of parts of a template face shape, the plurality of parts of the template face shape corresponding to the plurality of sub-regions; and

tracking a face included in the input image based on the occlusion probabilities.

2. The face tracking method of claim 1 , wherein at least a portion of the face in the input image is occluded.

3. The face tracking method of claim 1 , wherein the detecting detects the face region in the input image based on a previous tracking result.

4. The face tracking method of claim 1 , wherein the detecting comprises:

extracting a plurality of first feature points from a current frame of the input image;

selecting at least one key frame from a database;

estimating a pose of the face based on the plurality of first feature points and a plurality of second feature points of the at least one key frame; and

estimating a plurality of third feature points of the face based on the estimated pose.

5. The face tracking method of claim 4 , wherein the at least one key frame is indexed by a pose parameter and an expression parameter.

6. The face tracking method of claim 4 , wherein the selecting selects the at least one key frame based on a pose parameter and an expression parameter of a previous frame of the input image.

7. The face tracking method of claim 4 , wherein the at least one key frame includes three-dimensional (3D) coordinates of previously matched feature points, and feature vectors of the previously matched feature points.

8. The face tracking method of claim 4 , wherein the estimating the pose comprises:

generating matching relationship information associated with a matching relationship between the plurality of first feature points and the plurality of second feature points based on similarity between feature vectors of the plurality of first feature points and feature vectors of the plurality of second feature points; and

estimating a pose parameter based on a distance between coordinates of a first feature point and projected coordinates of a second feature point matched to the first feature point, the pose parameter indicating the pose of the face.

9. The face tracking method of claim 8 , wherein the detecting further comprises generating a valid matching result based on the distance.

10. The face tracking method of claim 1 , wherein the segmenting comprises:

generating a plurality of patches based on positions and colors of pixels included in the face region; and

generating a plurality of sections based on a plurality of feature points estimated from the face region.

11. The face tracking method of claim 10 , wherein the generating the plurality of sections comprises:

merging patches, from among the plurality of patches, that are adjacent to each of the estimated feature points; and

generating the plurality of sections.

12. The face tracking method of claim 1 , wherein

the occlusion probabilities include first occlusion probabilities for a plurality of patches and second occlusion probabilities for a plurality of sections;

the probability models include first probability models for the plurality of patches and second probability models for the plurality of sections; and

the calculating includes

calculating the first occlusion probabilities for the plurality of patches based on the first probability models for the plurality of patches,

calculating the second occlusion probabilities for the plurality of sections based on the second probability models for the plurality of sections, and

generating an occlusion weight map based on the first occlusion probabilities and the second occlusion probabilities.

13. The face tracking method of claim 12 , wherein the first probability models are designated for parts of the template face shape that correspond to patches among the plurality of parts of the template face shape.

14. The face tracking method of claim 12 , wherein a feature descriptor of the plurality of patches includes a feature associated with colors of pixels included in the plurality of patches.

15. The face tracking method of claim 12 , wherein the second probability models are associated with components corresponding to sections, from among the plurality of sections, that correspond to main components of the face.

16. The face tracking method of claim 12 , wherein a feature descriptor of the plurality of sections includes a feature associated with a number of valid matching results included in the plurality of sections.

17. The face tracking method of claim 12 , wherein the occlusion weight map includes an occlusion probability for each pixel in the face region.

18. The face tracking method of claim 1 , wherein the tracking comprises:

adjusting a parameter of a face model representing the face based on an occlusion weight map.

19. The face tracking method of claim 18 , wherein the face model includes at least one of a two-dimensional (2D) shape model, a 3D shape model and a texture model.

20. The face tracking method of claim 18 , wherein the adjusting adjusts the parameter of the face model to minimize a cost function defined based on the occlusion weight map.

21. The face tracking method of claim 1 , further comprising:

evaluating a tracking result using a trained classifier; and

updating a key frame when the tracking result is evaluated as successful.

22. The face tracking method of claim 21 , further comprising:

determining whether occlusion occurs in the input image when the tracking result is evaluated as successful; and

updating a probability model, among the probability models, when the occlusion is determined not to occur.

23. A non-transitory computer readable recording medium storing a program that, when executed on a computer, causes the computer to implement the method of claim 1 .

24. A face tracking apparatus, comprising:

at least one processor configured to executed computer-readable instructions to

detect a face region based on an input image;

segment the face region into a plurality of sub-regions;

calculate occlusion probabilities for the plurality of sub-regions, the occlusion probabilities for the plurality of sub-regions calculated based on probability models for a plurality of parts of a template face shape, the plurality of parts of the template face shape corresponding to the plurality of sub-regions; and

track a face included in the input image based on the occlusion probabilities.

25. The face tracking apparatus of claim 24 , wherein at least a portion of the face in the input image is occluded.

26. The face tracking apparatus of claim 24 , wherein the at least one processor is further configured to execute the computer-readable instructions to

extract a plurality of first feature points from a current frame of the input image;

select at least one key frame from a database;

estimate a pose of the face based on the plurality of first feature points and a plurality of second feature points of the at least one key frame; and

estimate a plurality of third feature points of the face based on the estimated pose.

27. The face tracking apparatus of claim 26 , wherein:

the at least one key frame includes information associated with the plurality of second feature points; and

the at least key frame is indexed by a pose parameter and an expression parameter.

28. The face tracking apparatus of claim 24 , wherein the at least one processor is further configured to execute the computer-readable instructions to:

generate a plurality of patches based on positions and colors of pixels in the face region; and

generate a plurality of sections based on a plurality of feature points estimated from the face region.

29. The face tracking apparatus of claim 28 , wherein the at least one processor is further configured to execute the computer-readable instructions to

merge patches, from among the plurality of patches, that are adjacent to each of the plurality of estimated feature points; and

generate the plurality of sections.

30. The face tracking apparatus of claim 24 , wherein

the occlusion probabilities include first occlusion probabilities for a plurality of patches and second occlusion probabilities for a plurality of sections;

the probability models include first probability models for the plurality of patches and second probability models for the plurality of sections; and

the at least one processor is further configured to execute the computer-readable instructions to

determine the first probability models for the plurality of patches,

determine the second probability models for the plurality of sections;

calculate the first occlusion probabilities for the plurality of patches based on the first probability models;

calculate the second occlusion probabilities for the plurality of sections based on the second probability models; and

generate an occlusion weight map based on the first occlusion probabilities and the second occlusion probabilities.