System and Method for Detecting and Removing Occlusions in a Three-Dimensional Image
A system and method for performing facial recognition is described. In some implementations, the system and method identify points of a three-dimensional scan that are associated with occlusions, such as eyeglasses, to a face of a target subject and remove the identified points from the three-dimensional scan.
1 . A method for removing an occlusion from a three-dimensional scan of a target subject, the method comprising:
receiving the three-dimensional scan that includes a face of the target subject, the three-dimensional scan comprising a plurality of three-dimensional points;
determining that the plurality of three-dimensional points includes a subset plurality of three-dimensional points corresponding to the occlusion, the subset plurality of three-dimensional points corresponding to a subset of the plurality of three-dimensional points; and
removing the subset plurality of three-dimensional points corresponding to the occlusion of the face of the target subject from the three-dimensional scan.
2 . The method of claim 1 , further comprising reconstructing the three-dimensional scan to compensate for the removed subset plurality of three-dimensional points.
3 . The method of claim 1 , further comprising comparing the three-dimensional scan with the removed subset plurality of three-dimensional points against a known image.
4 . The method of claim 1 , wherein removing the subset plurality of three-dimensional points corresponding to the occlusion of the face comprises:
converting the three-dimensional scan to a two-dimensional depth image;
detecting edge points in the two-dimensional depth image;
determining edge points in the three-dimensional scan based on the detected edge points in the two-dimensional scan; and
filtering the edge points in the three-dimensional scan to identify those edge points inside a particular region of the three-dimensional scan associated with the occlusion.
5 . The method of claim 4 , wherein filtering the edge points in the three-dimensional scan to identify those edge points inside a particular region of the three-dimensional scan associated with the occlusion comprises filtering the edge points in the three-dimensional scan to identify those edge points inside an ocular portion of the three-dimensional scan.
6 . The method of claim 5 , wherein removing the subset plurality of three-dimensional points corresponding to the occlusion of the face further comprises:
fitting an occlusion model to the filtered edge points to identify those edge points corresponding to potential eyeglasses points.
7 . The method of claim 6 , wherein fitting an occlusion model to the filtered edge points to identify those edge points corresponding to potential eyeglasses points comprises:
determining a first boundary that is a first predetermined distance from the occlusion model;
determining a second boundary that is a second predetermined distance from the occlusion model, wherein the first boundary and the second boundary are on opposite sides of the occlusion model; and
identifying those edge points that lie on either of the first boundary or the second boundary or that lie in between the first boundary and the second boundary.
8 . The method of claim 7 , wherein the first predetermined distance is the same as the second predetermined distance but in an opposite direction.
9 . The method of claim 6 , further comprising:
fitting an ellipse to each ocular region; and
identifying those edge points that fall within either ellipses corresponding to the left and right ocular regions.
10 . The method of claim 6 , further comprising:
eliminating those edge points identified as fitting the occlusion model.
11 . The method of claim 9 , further comprising:
eliminating those edge points identified as falling within either ellipse.
12 . The method of claim 9 , further comprising:
eliminating those edge points identified as both fitting the occlusion model and falling within either ellipse.
13 . A system for removing an occlusion from a three-dimensional scan of a target subject, the system comprising:
a 3D sensor configured to acquire a three-dimensional scan of the target subject, the three-dimensional scan of the target including a face of the target subject, the three-dimensional scan comprising at least a range to each of a plurality of points on the target subject; and
a processor configured to:
receive the three-dimensional scan from the 3D sensor,
determine that the three-dimensional scan includes an occlusion to the face of the target subject,
remove, from the three-dimensional scan, a plurality of three-dimensional points corresponding to the occlusion of the face of the target subject.
14 . The system of claim 13 , wherein the occlusion comprises a hand, a pair of eyeglasses, a hat, a scarf, a cellphone, hair, or a cup.
15 . The system of claim 13 , wherein the processor is further configured to reconstruct the three-dimensional scan to compensate for the removed plurality of three-dimensional points corresponding to the occlusion of the face of the target subject.
16 . The system of claim 13 , wherein the processor is further configured to compare the three-dimensional scan with the removed plurality of three-dimensional points against a known image.
17 . The system of claim 13 , wherein the processor is further configured to compare the reconstructed three-dimensional scan against a known image.
18 . The system of claim 13 , wherein the processor configured to remove, from the three-dimensional scan, a plurality of three-dimensional points corresponding to the occlusion of the face of the target subject comprises the processor configured to:
convert the three-dimensional scan to a two-dimensional depth image;
detect edge points in the two-dimensional depth image;
determine edge points in the three-dimensional scan based on the detected edge points in the two-dimensional scan; and
filter the edge points in the three-dimensional scan to identify those edge points inside a particular region of the three-dimensional scan associated with the occlusion.
19 . The system of claim 18 , wherein the processor configured to remove, from the three-dimensional scan, a plurality of three-dimensional points corresponding to the occlusion of the face of the target subject comprises the processor configured to:
fit an occlusion model to the filtered edge points;
fit an ellipse to the filtered edge points for each ocular region; and
eliminate, from the three-dimensional scan, those edge points that both fit the occlusion model and fit either of the ellipses.
20 . The system of claim 19 , wherein the processor configured to fit an occlusion model to the filtered edge points comprises the processor configured to:
determine a first boundary that is a first predetermined distance from the occlusion model;
determine a second boundary that is a second predetermined distance from the occlusion model, wherein the first boundary and the second boundary are on opposite sides of the occlusion model; and
identify those edge points that lie on either of the first boundary or the second boundary or that lie in between the first boundary and the second boundary.