Method for determining face-pose from facial features and physical structure
View Patent ↗An invention is here disclosed for a method of determining human face-pose in visual images using a mathematical model composed of equations for the mechanics of the physics relating the orientation and position of the facial pose to the facial features in the image, wherein the mechanics are a function of the face physical structure. In particular, the orientation is computed from facial feature images with equations based on the physical offset between the face and eye-orbit margins considered as separate anatomically located facial planes, where the offset is a fixed quantity set by the physical structure. The pose position is computed from the facial structure; a fixed-point iteration process is used to improve the computation to correct for facial front offset. The method may be readily used for real-time determination of face-pose in benign webcam applications such as locating work-areas that the user is physically facing in a workspace.
1 . A method for determining a face pose in an image
a) isolating a face in the image, determining a facial image bounding frame for a spatial plane containing facial features imaged in the isolated face image, and determining a geometric center of the facial bounding frame;
b) detecting eye orbits of the face, determining an eye image bounding frame for a spatial plane containing eye orbit features imaged in an eye orbital image, determining a geometric center of the eye bounding frame, determining physical size of orbit margins of the eye orbits, and setting eye orbital center points for each eye orbit;
c) compute a camera depth of the isolated face location using a separation distance between the detected eye orbits and the orbit margins;
d) computing Euler angles of yaw, pitch, and roll offsets from image plane reference axes wherein:
roll is computed from lateral and vertical displacements of the eye orbital center points;
yaw is computed from the lateral difference between the geometric center of the eye bounding frame and the facial bounding frame, and from the camera depth;
pitch is computed from a vertical difference between the geometric center of the eye bounding frame and the facial bounding frame and from the camera depth; and
e) determining the face pose of the image based on the camera depth, and the computed Euler angles of yaw, pitch and roll.
2 . The method of claim 1 , wherein spatial planes for an eye orbit image and a face image are determined from anatomically located facial planar surfaces for the eye orbit and for the face, wherein the facial planar surfaces are delineated by a set of physical features of the physical structure of the face, and the facial planar surfaces are physically separated.
3 . The method of claim 2 , wherein:
a set of physical features of a facial planar surface for an eye orbit image are those of a superior orbital region and the orbital cavities of the eyes;
a set of physical features of a facial planar surface for a full face image are those of the anterior facial shape periphery, superior forehead, and inferior face;
a set of physical features of a facial plane surface for a central face image are those of an outer canthus of the eyes and a nares of the nose.
4 . The method of claim 3 , wherein frame coordinates are computed for image bounding frames containing image coordinates of delineating sets of physical features of the face physical structure for the facial planar surfaces, wherein:
geometric centers are computed for the image bounding frames from the frame coordinates.
5 . The method of claim 4 , wherein the yaw is computed from the lateral difference between the geometric center of the image bounding frame for the eye orbit image of the facial planar surface for the eye orbit, and the geometric center of the image bounding frame for the face image of the facial planar surface for the face, wherein:
the computations of yaw are functions of the camera depth and of the physical separation between the facial planar surface for the face and the facial planar surface for the eye orbit.
6 . The method of claim 4 , wherein the pitch is computed from a vertical difference between the geometric center of the image bounding frame for the eye orbit image of the facial planar surface for the eye orbit, and the geometric center of the image bounding frame for the face image of the facial planar surface for the face, wherein:
the computation of pitch is a function of the camera depth and of the physical separation between a facial planar surface for the face and the facial planar surface for the eye orbit.
7 . The method of claim 1 , wherein a fixed-point iteration process is used to refine the computation of the camera depth from the computed Euler yaw, pitch, and roll angles, wherein
the process is repeated for convergence of the depth to within a set tolerance.
8 . An apparatus to determine a face pose in an image comprising:
a processor; and a memory, including instructions stored thereon, which when executed by the processor cause the apparatus to:
a) isolate a face in the image, determine a facial image bounding frame for a spatial plane containing facial features imaged in the isolated face image, and determining a geometric center of the facial bounding frame;
b) detect eye orbits of the face, determine an eye image bounding frame for a spatial plane containing eye orbit features imaged in an eye orbital image, determine a geometric center of the eye bounding frame, determine physical size of orbit margins of the eye orbits, and setting eye orbital center points for each eye orbit;
c) compute a camera depth of the isolated face location using a separation distance between the detected eye orbits and the orbit margins;
d) computing Euler angles of yaw, pitch, and roll offsets from image plane reference axes wherein:
roll is computed from lateral and vertical displacements of the eye orbital center points;
yaw is computed from the lateral difference between the geometric center of the eye bounding frame and the facial bounding frame, and from the camera depth;
pitch is computed from a vertical difference between the geometric center of the eye bounding frame and the facial bounding frame, and from the camera depth; and
e) determining the face pose of the image based on the camera depth, and the computed Euler angles of yaw, pitch and roll.
9 . The apparatus of claim 8 , wherein spatial planes for an eye orbit image and a face image are determined from anatomically located facial planar surfaces for the eye orbit and for the face, wherein the facial planar surfaces are delineated by a set of physical features of the physical structure of the face, and the facial planar surfaces are physically separated.
10 . The apparatus of claim 9 , wherein:
a set of physical features of a facial planar surface for an eye orbit image are those of a superior orbital region and the orbital cavities of the eyes;
a set of physical features of a facial planar surface for a full face image are those of the anterior facial shape periphery, superior forehead, and inferior face;
a set of physical features of a facial plane surface for a central-face image are those of an outer canthus of the eyes and a nares of the nose.
11 . The apparatus of claim 10 , wherein frame coordinates are computed for image bounding frames containing image coordinates of delineating sets of physical features of the face physical structure for the facial planar surfaces, wherein:
geometric centers are computed for the image bounding frames from the frame coordinates.
12 . The apparatus of claim 11 , wherein the yaw is computed from the lateral difference between the geometric center of the image bounding frame for the eye orbit image of the facial planar surface for the eye orbit, and the geometric center of the image bounding frame for the face image of the facial planar surface for the face, wherein:
the computations of yaw are functions of the camera depth and of the physical separation between the facial planar surface for the face and the facial planar surface for the eye orbit.
13 . The apparatus of claim 11 , wherein the pitch is computed from a vertical difference between the geometric center of the image bounding frame for the eye orbit image of the facial planar surface for the eye-orbit, and the geometric center of the image bounding frame for the face image of the facial planar surface for the face, wherein:
the computation of pitch is a function of the camera depth and of the physical separation between the facial planar surface for the face and the facial planar surface for the eye orbit.
14 . The apparatus of claim 8 , wherein a fixed-point iteration process is used to refine the computation of the camera depth from the computed Euler yaw, pitch, and roll angles, wherein the process is repeated for convergence of the depth to within a set tolerance.
15 . A non-transitory computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out determining a face pose in an image, by:
a) isolating a face in the image, determining a facial image bounding frame for a spatial plane containing facial features imaged in the isolated face image, and determining a geometric center of the facial bounding frame;
b) detecting eye orbits of the face, determining an eye image bounding frame for a spatial plane containing eye orbit features imaged in an eye orbital image, determining a geometric center of the eye bounding frame, determining physical size of orbit margins of the eye orbits, and setting eye orbital center points for each eye orbit;
c) compute a camera depth of the isolated face location using a separation distance between the detected eye orbits and the orbit margins;
d) computing Euler angles of yaw, pitch, and roll offsets from image plane reference axes wherein:
roll is computed from lateral and vertical displacements of the eye orbital center points;
yaw is computed from the lateral difference between the geometric center of the eye bounding frame and the facial bounding frame, and from the camera depth;
pitch is computed from a vertical difference between the geometric center of the eye bounding frame and the facial bounding frame, and from the camera depth; and
e) determining the face pose of the image based on the camera depth, and the computed Euler angles of yaw, pitch and roll.
16 . The non-transitory computer-readable storage medium of claim 15 , wherein spatial planes for an eye orbit image and a face image are determined from anatomically located facial planar surfaces for the eye orbit and for the face, wherein the facial planar surfaces are delineated by a set of physical features of the physical structure of the face, and the facial planar surfaces are physically separated.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein:
a set of physical features of a facial planar surface for an eye orbit image are those of a superior orbital region and the orbital cavities of the eyes;
a set of physical features of a facial planar surface for a full face image are those of the anterior facial shape periphery, superior forehead, and inferior face;
a set of physical features of a facial plane surface for a central-face image are those of an outer canthus of the eyes and a nares of the nose.
18 . The non-transitory computer-readable storage medium of claim 17 , wherein frame coordinates are computed for image bounding frames containing image coordinates of delineating sets of physical features of the face physical structure for the facial planar surfaces, wherein:
geometric centers are computed for the image bounding frames from the frame coordinates.
19 . The non-transitory computer-readable storage medium of claim 18 , wherein the yaw is computed from the lateral difference between the geometric center of the image bounding frame for the eye orbit image of the facial planar surface for the eye orbit, and the geometric center of the image bounding frame for the face image of the facial planar surface for the face, wherein:
the computations of yaw are functions of the camera depth and of the physical separation between the facial planar surface for the face and the facial planar surface for the eye orbit.
20 . The non-transitory computer-readable storage medium of claim 18 , wherein the pitch is computed from a vertical difference between the geometric center of the image bounding frame for the eye orbit image of the facial planar surface for the eye orbit, and the geometric center of the image bounding frame for the face image of the facial planar surface for the face, wherein:
the computation of the pitch is a function of the camera depth and of the physical separation between the facial planar surface for the face and the facial planar surface for the eye orbit.