System and Method for Off Angle Three-Dimensional Face Standardization for Robust Performance
A system uses range and Doppler velocity measurements from a lidar system and images from a video system to estimate a six degree-of-freedom trajectory of a target. The system utilizes a two-stage solution to obtain 3D standardized face representations from non-frontal face views for a statistical learning algorithm. The first stage standardizes the pose (non-frontal 3D face representation) to a frontal view and the second stage uses facial symmetry to fill in missing facial regions due to yaw face pose variations (i.e. rotation about the y-axis).
1 . A system for obtaining 3D standardized face representations from non-frontal face views, the system comprising:
a lidar subsystem configured to direct at least two beams toward the target and generate a plurality of three-dimensional (3D) measurements for a plurality of points on the target for each of the at least two beams;
a video subsystem configured to provide a plurality of two-dimensional images of the target; and
a processor configured to:
receive, from the lidar subsystem, the plurality of 3D measurements for the plurality of points on the target,
receive, from the video subsystem, the plurality of 2D images of the target,
generate at least one 3D image of the target based on the 3D measurements for the plurality of points on the target and the 2D images of the target,
determine a non-frontal 3D face representation based on the at least one 3D image,
transform the non-frontal 3D face representation to a frontal 3D face representation,
generate a mirror face representation of the frontal 3D face representation, and
generate a 3D standardized face representation by filling one or more missing regions in the frontal 3D face representation based on data from the mirror face representation.
2 . The system of claim 1 , wherein the processor configured to transform the non-frontal 3D face representation to the frontal 3D face representation is further configured to:
determine a plurality of candidate face anchor points on the non-frontal 3D face representation;
determine a first anchor point region for each of the plurality of candidate face anchor points;
determine a quality of alignment between the first anchor point region and a reference anchor point region associated with a reference 3D face representation;
determine a face anchor point from the plurality of candidate face anchor points based on the quality of alignment; and
translate the non-frontal 3D face representation to collocate the determined face anchor point with a reference anchor point associated with the reference 3D face representation and transform it to a frontal orientation using an iterative registration process to a frontal face reference model.
3 . The system of claim 1 , wherein the one or more missing regions comprises at least one hole in the frontal 3D face representation, and wherein the processor is further configured to:
for each of a plurality of interpolated points in the at least one hole, determine a closest surface point on the mirror face representation; and
merge the determined closest surface point into the frontal 3D face representation.
4 . The system of claim 1 , wherein the one or more missing regions comprises at least one eroded face boundary in the frontal 3D face representation, and wherein the processor is further configured to:
trace a right or left boundary of the frontal 3D face representation and a right or left boundary of the mirror face representation;
determine a region that is to be extracted from the mirror face representation and merged with the frontal 3D face representation based on the traced boundaries; and
merge a plurality of points in the determined region of the mirror face representation with the frontal 3D face representation.