Face image processing device and face image processing program
A face image processing device, includes: an image coordinate system coordinate value derivation unit detecting an x-coordinate value and a y-coordinate value in an image coordinate system at a feature point of an organ of a face of a person in an image, and estimating a z-coordinate value, so as to derive three-dimensional coordinate values in the image coordinate system; a camera coordinate system coordinate value derivation unit deriving three-dimensional coordinate values in a camera coordinate system from the three-dimensional coordinate values in the image coordinate system derived by the image coordinate system coordinate value derivation unit; and a parameter derivation unit applying the three-dimensional coordinate values in the camera coordinate system derived by the camera coordinate system coordinate value derivation unit to a predetermined three-dimensional face shape model to derive a model parameter of the three-dimensional face shape model in the camera coordinate system.
1. A face image processing device, comprising:
at least one processor configured to implement:
an image coordinate system coordinate value derivation unit configured to detect an x-coordinate value which is a horizontal coordinate value and a y-coordinate value which is a vertical coordinate value respectively in an image coordinate system at a feature point of an organ of a face of a person in an image acquired by imaging the face, and estimate a z-coordinate value which is a depth coordinate value in the image coordinate system, so as to derive three-dimensional coordinate values in the image coordinate system;
a camera coordinate system coordinate value derivation unit configured to derive three-dimensional coordinate values in a camera coordinate system from the three-dimensional coordinate values in the image coordinate system derived by the image coordinate system coordinate value derivation unit; and
a parameter derivation unit configured to apply the three-dimensional coordinate values in the camera coordinate system derived by the camera coordinate system coordinate value derivation unit to a predetermined three-dimensional face shape model to derive a model parameter of the three-dimensional face shape model in the camera coordinate system.
2. The face image processing device according to claim 1 , wherein
the image coordinate system coordinate value derivation unit is configured to derive the z-coordinate value by estimating the z-coordinate value using deep learning in parallel with the detection of the x-coordinate value and the y-coordinate value.
3. The face image processing device according to claim 1 , wherein
the three-dimensional face shape model is formed of a linear sum of an average shape and a basis, and the basis is separated into an individual difference basis, which is a component that does not change with time, and a facial expression basis, which is a component that changes with time, and
the model parameter includes a parameter of the individual difference basis and a parameter of the facial expression basis.
4. The face image processing device according to claim 3 , wherein
the model parameter further includes a position and orientation parameter indicating a position and an orientation of the face of the person as viewed from a camera that images the face of the person.
5. The face image processing device according to claim 1 , further comprising:
a face distance estimation unit configured to estimate a distance to the face of the person, wherein
the camera coordinate system coordinate value derivation unit is configured to derive the z-coordinate value in the camera coordinate system by using the distance estimated by the face distance estimation unit.
6. The face image processing device according to claim 1 , further comprising:
a coordinate correction unit configured to correct the z-coordinate value, which is derived by the image coordinate system coordinate value derivation unit, according to a position of the face in the image.
7. The face image processing device according to claim 1 , further comprising:
a weight setting unit configured to set independent weights for respective error evaluation values of the x-coordinate value, the y-coordinate value, and the z-coordinate value at each feature point of each organ of the face.
8. A face image processing program stored in a non-transitory computer readable medium configured to cause a computer to execute processing, the processing comprising:
detecting an x-coordinate value which is a horizontal coordinate value and a y-coordinate value which is a vertical coordinate value respectively in an image coordinate system at a feature point of an organ of a face of a person in an image acquired by imaging the face, and estimating a z-coordinate value which is a depth coordinate value in the image coordinate system, so as to derive three-dimensional coordinate values in the image coordinate system;
deriving three-dimensional coordinate values in a camera coordinate system from the derived three-dimensional coordinate values in the image coordinate system; and
applying the derived three-dimensional coordinate values in the camera coordinate system to a predetermined three-dimensional face shape model to derive a model parameter of the three-dimensional face shape model in the camera coordinate system.