Image processing method and information processing device

1. An information processing device, comprising:

a processor configured to:

construct a three-dimensional face model based on a two-dimensional shape of a face object in a two-dimensional image, said two-dimensional shape comprising a plurality of key points;

determine a correspondence between the constructed three-dimensional face model and the face object in the two-dimensional image;

for each of the plurality of key points, determine, based on the correspondence, an area of interest of a key point in the two-dimensional image using an area of interest near a corresponding key point in the three-dimensional face model, and extract an image feature from the determined area of interest in the two-dimensional image as the image feature of the key point in the two-dimensional image;

establish, based on the extracted image features of the key points, a current increment of the two-dimensional shape of the face object using a regressor which is obtained in advance, said increment indicating a displacement adjustment to be performed on positions of the key points comprised in the two-dimensional shape; and

obtain a current two-dimensional shape of the face object based on a previously obtained two-dimensional shape of the face object and the current increment of the two-dimensional shape of the face object,

wherein the processor is further configured to iteratively perform steps of constructing a respective three-dimensional face model, determining a respective correspondence, determining a respective area of interest, extracting a respective image feature, establishing a respective current increment and obtaining a respective current two-dimensional shape, until a predetermined iteration termination condition is satisfied.

2. The information processing device according to claim 1 , wherein the key points comprise canthi, eyebrows, a nose tip, mouth corners and/or a chin center.

3. The information processing device according to claim 1 , wherein the regressor is obtained through training based on two-dimensional training images including face objects in which two-dimensional shapes are marked in advance.

4. The information processing device according to claim 1 , wherein the processor is further configured to determine the area of interest in the two-dimensional image by:

projecting the area of interest in the three-dimensional face model to the two-dimensional image according to the correspondence, and taking a projecting result as the area of interest in the two-dimensional image.

5. The information processing device according to claim 4 , wherein the area of interest near the corresponding key point in the three-dimensional face model comprises a portion of the three-dimensional face model that is within a sphere centered on the corresponding key point in the three-dimensional face model.

6. The information processing device according to claim 4 , wherein the processor is further configured to:

determine a three-dimensional posture of the face object in determining the correspondence between the three-dimensional face model and the face object in the two-dimensional object; and

judge, in determining the area of interest in the two-dimensional image, whether the corresponding key point in the three-dimensional face model would be visible in the two-dimensional image according to the determined three-dimensional posture, wherein the projecting is performed only for a particular area of interest near a particular key point that would be visible in the two-dimensional image.

7. An image processing method, comprising:

constructing a three-dimensional face model based on a two-dimensional shape of a face object in a two-dimensional image, said two-dimensional shape comprising a plurality of key points;

determining a correspondence between the constructed three-dimensional face model and the face object in the two-dimensional image;

for each of the plurality of key points, determining, based on the correspondence, an area of interest of a key point in the two-dimensional image using an area of interest near a corresponding key point in the three-dimensional face model, and extracting an image feature from the determined area of interest in the two-dimensional image as the image feature of the key point in the two-dimensional image;

establishing, based on the extracted image features of the key points, a current increment of the two-dimensional shape of the face object using a regressor which is obtained in advance, said increment indicating a displacement adjustment to be performed on positions of the key points comprised in the two-dimensional shape; and

obtaining a current two-dimensional shape of the face object based on a previously obtained two-dimensional shape of the face object and the current increment of the two-dimensional shape of the face object,

wherein steps of constructing a respective three-dimensional face model, determining a respective correspondence, determining a respective area of interest, extracting a respective image feature, establishing a respective current increment and obtaining a respective current two-dimensional shape are performed iteratively, until a predetermined iteration termination condition is satisfied.

8. The image processing method according to claim 7 , wherein the key points include canthi, eyebrows, a nose tip, mouth corners and/or a chin center.

9. The image processing method according to claim 7 , wherein the regressor is obtained through training based on two-dimensional training images including face objects in which two-dimensional shapes are marked in advance.

10. The image processing method according to claim 7 , wherein determining the area of interest of the key point in the two-dimensional image using the area of interest near the corresponding key point in the three-dimensional face model includes:

projecting the area of interest in the three-dimensional face model to the two-dimensional image according to the correspondence, and taking a projecting result as the area of interest in the two-dimensional image.

11. The image processing method according to claim 10 , wherein the area of interest near the corresponding key point in the three-dimensional face model includes a portion of the three-dimensional face model that is within a sphere centered on the corresponding key point in the three-dimensional face model.

12. The image processing method according to claim 10 , wherein

determining the correspondence between the three-dimensional face model and the face object in the two-dimensional image includes: determining a three-dimensional posture of the face object, and

determining the area of interest of the key point in the two-dimensional image using the area of interest near the corresponding key point in the three-dimensional face model further includes: judging whether the corresponding key point in the three-dimensional face model would be visible in the two-dimensional image according to the determined three-dimensional posture,

wherein the projecting is performed only for a particular area of interest near a particular key point that would be visible in the two-dimensional image.

13. A non-transitory recording medium storing a program, wherein the program, when executed by a computer, causes the computer to perform an image processing method, the method including:

constructing a three-dimensional face model based on a two-dimensional shape of a face object in a two-dimensional image, said two-dimensional shape comprising a plurality of key points;

determining a correspondence between the constructed three-dimensional face model and the face object in the two-dimensional image;

for each of the plurality of key points, determining, based on the correspondence, an area of interest of a key point in the two-dimensional image using an area of interest near a corresponding key point in the three-dimensional face model, and extracting an image feature from the determined area of interest in the two-dimensional image as the image feature of the key point in the two-dimensional image;

establishing, based on the extracted image features of the key points, a current increment of the two-dimensional shape of the face object using a regressor which is obtained in advance, said increment indicating a displacement adjustment to be performed on positions of the key points comprised in the two-dimensional shape; and

obtaining a current two-dimensional shape of the face object based on a previously obtained two-dimensional shape of the face object and the current increment of the two-dimensional shape of the face object,

wherein steps of constructing a respective three-dimensional face model, determining a respective correspondence, determining a respective area of interest, extracting a respective image feature, establishing a respective current increment and obtaining a respective current two-dimensional shape are performed iteratively, until a predetermined iteration termination condition is satisfied.