Head pose estimation
A method and apparatus for estimating a user's head pose relative to a sensing device. The sensing device detects a face of the user in an image. The sensing device further identifies a plurality of points in the image corresponding to respective features of the detected face. The plurality of points includes at least a first point corresponding to a location of a first facial feature. The sensing device determines a position of the face relative to the sensing device based at least in part on a distance between the first point in the image and one or more of the remaining points. For example, the sensing device may determine a pitch, yaw, distance, or location of the user's face relative to the sensing device.
1. A method of determining a head pose of a user by a sensing device, comprising:
detecting a face of the user in a first image;
identifying a plurality of points in the first image corresponding to respective features of the detected face, the plurality of points including at least a first point corresponding to a location of a first facial feature, a second point corresponding to a location of a second facial feature, a third point corresponding to a location of a third facial feature, a fourth point corresponding to a location of a fourth facial feature, and a fifth point corresponding to a location of a fifth facial feature;
calculating a first midpoint in the first image between the location of the second facial feature and the location of the third facial feature;
calculating a second midpoint in the first image between the location of the fourth facial feature and the location of the fifth facial feature;
calculating a third midpoint in the first image between the location of the second facial feature and the location of the fourth facial feature;
calculating a fourth midpoint in the first image between the location of the third facial feature and the location of the fifth facial feature;
calculating a first combined distance from the first point to each of the first midpoint and the second midpoint;
calculating a second combined distance from the first point to each of the third midpoint and the fourth midpoint; and
determining a position of the face relative to the sensing device based at least in part on one or more second images of the user, the greater of the first combined distance or the second combined distance, and a ratio of the distances between the first point and each of the first and second midpoints or a ratio of the distances between the first point and each of the third and fourth midpoints.
2. The method of claim 1 , wherein the first facial feature is a nose and the remaining points correspond to locations of a mouth or eyes.
3. The method of claim 2 , further comprising:
determining a yaw of the face relative to the sensing device based on the ratio of the distances between the first point and each of the first and second midpoints or the ratio of the distances between the first point and each of the third and fourth midpoints.
4. The method of claim 3 , wherein
the second facial feature is a left eye, the fourth facial feature is a left portion of a mouth,
the third facial feature is a right eye, and the fifth facial feature is a right portion of the mouth.
5. The method of claim 1 , further comprising:
determining a pitch of the face relative to the sensing device based on the ratio of the distances between the first point and each of the first and second midpoints or the ratio of the distances between the first point and each of the third and fourth midpoint.
6. The method of claim 5 , wherein
second facial feature is a left eye, the third facial feature is a right eye,
the fourth facial feature is a left portion of a mouth, and the fifth facial feature is a right portion of the mouth.
7. The method of claim 1 , wherein
the second facial feature is a left eye, the third facial feature is a right eye,
the fourth facial feature is a left portion of a mouth, and the fifth facial feature is a right portion of the mouth.
8. The method of claim 1 , wherein the determining of the position of the face comprises:
determining that the user is in contact with the sensing device; and
determining a scalar quantity based on the one or more second images of the user, the position of the face being determined based at least in part on the scalar quantity.
9. A sensing device, comprising:
processing circuitry; and
memory storing instructions that, when executed by the processing circuitry, causes the sensing device to:
detect a face in a first image;
identify a plurality of points in the first image corresponding to respective features of the detected face, the plurality of points including at least a first point corresponding to a location of a first facial feature, a second point corresponding to a location of a second facial feature, a third point corresponding to a location of a third facial feature, a fourth point corresponding to a location of a fourth facial feature, and a fifth point corresponding to a location of a fifth facial feature;
calculate a first midpoint in the first image between the location of the second facial feature and the location of the third facial feature;
calculate a second midpoint in the first image between the location of the fourth facial feature and the location of the fifth facial feature;
calculate a third midpoint in the first image between the location of the second facial feature and the location of the fourth facial feature;
calculate a fourth midpoint in the first image between the location of the third facial feature and the location of the fifth facial feature;
calculate a first combined distance from the first point to each of the first midpoint and the second midpoint;
calculate a second combined distance from the first point to each of the third midpoint and the fourth midpoint; and
determine a position of the face relative to the sensing device based at least in part on one or more second images of the user, the greater of the first combined distance or the second combined distance, and a ratio of the distances between the first point and each of the first and second midpoints or a ratio of the distances between the first point and each of the third and fourth midpoints.
10. The sensing device of claim 9 , wherein the first facial feature is a nose and the remaining points correspond to locations of a mouth or eyes.
11. The sensing device of claim 10 , wherein
the second facial feature is a left eye, the third facial feature is a right eye,
the fourth facial feature is a left portion of a mouth, and the fifth facial feature is a right portion of the mouth.
12. The sensing device of claim 9 , wherein
the second facial feature is a left eye, the fourth facial feature is a left portion of a mouth,
the third facial feature is a right eye, and the fifth facial feature is a right portion of the mouth.
13. The sensing device of claim 9 , wherein execution of the instructions for determining the position of the face causes the sensing device to:
determine that the user is in contact with the sensing device; and
determine a scalar quantity based on the one or more second images of the user, the position of the face being determined based at least in part on the scalar quantity.
14. A system, comprising,
a camera configured to capture a first image of a scene; and
a sensing device configured to:
detect a face in the first image;
identify a plurality of points in the first image corresponding to respective features of the detected face, the plurality of points including at least a first point corresponding to a location of a first facial feature, a second point corresponding to a location of a second facial feature, a third point corresponding to a location of a third facial feature, a fourth point corresponding to a location of a fourth facial feature, and a fifth point corresponding to a location of a fifth facial feature;
calculating a first midpoint in the first image between the location of the second facial feature and the location of the third facial feature;
calculating a second midpoint in the first image between the location of the fourth facial feature and the location of the fifth facial feature;
calculating a third midpoint in the first image between the location of the second facial feature and the location of the fourth facial feature;
calculating a fourth midpoint in the first image between the location of the third facial feature and the location of the fifth facial feature;
calculating a first combined distance from the first point to each of the first midpoint and the second midpoint;
calculating a second combined distance from the first point to each of the third midpoint and the fourth midpoint; and
determine a position of the face relative to the camera based at least in part on the one or more second images, the greater of the first combined distance or the second combined distance, and a ratio of the distances between the first point and each of the first and second midpoints or a ratio of the distances between the first point and each of the third and fourth midpoints.
15. The system of claim 14 , wherein
the second facial feature is a left eye, the third facial feature is a right eye,
the fourth facial feature is a left portion of a mouth, and the fifth facial feature is a right portion of the mouth, and the sensing device is to determine the position of the face by:
determining a pitch of the face relative to the camera based on the ratio of the distances from the first point to each of the first and second midpoints.
16. The system of claim 14 , wherein
the second facial feature is a left eye, and the fourth facial feature is a left portion of a mouth,
the third facial feature is a right eye, and the fifth facial feature is a right portion of the mouth, and the sensing device is to determine the position of the face by:
determining a yaw of the face relative to the camera based on the ratio of the distances from the first point to each of the third and fourth midpoints.