Method and system of multi-pass iterative closest point (ICP) registration in automated facial reconstruction

1. A computerized method for implementing a multi-pass iterative closest point (ICP) registration in an automated facial reconstruction process comprising:

providing an automated facial reconstruction process;

detecting that a face tracking application programming interface (API) misalignment has occurred during the automated facial reconstruction process;

implementing a multi-pass ICP process for the face tracking API misalignment by:

obtaining a first point cloud (PC 1 ) of the automated facial reconstruction process;

obtaining a second point cloud (PC 2 ) of the automated facial reconstruction process;

obtaining a first face mask (mask 1 ) of the automated facial reconstruction process;

obtaining a second face mask (mask 2 ) of the automated facial reconstruction process;

performing a first point cloud (PC 1 )-to-mask ICP operation by inputting and utilizing PC 1 and mask 1 , wherein a transformation output of first PC-to-mask ICP operation is provided to a third PC-to-PC ICP and a fourth PC-to-PC ICP;

performing a second PC-to-mask ICP on PC 2 and mask 2 , wherein a transformation output of the second PC-to-mask ICP step is provided to a second PC-to-PC ICP and to the third PC-to-PC ICP;

performing a first PC-to-PC ICP on the PC 1 and the PC 2 , wherein the first PC-to-PC ICP outputs a first transformation of the first PC-to-PC ICP and a first error value of the first PC-to-PC ICP;

performing the second PC-to-PC ICP on the transformation output by the second PC-to-mask ICP and the PC 1 , wherein the second PC-to-PC ICP outputs a second transformation of the second PC-to-PC ICP and a second error value of the second PC-to-PC ICP;

performing the third PC-to-PC ICP on the transformation output of the first PC-to-mask ICP and the transformation output of the second PC-to-mask ICP, wherein the third PC-to-PC ICP outputs a third transformation of the third PC-to-PC ICP and a third error value of the third PC-to-PC ICP; and

performing the fourth PC-to-PC ICP on the transformation output of the first PC-to-mask ICP and the PC 2 , wherein the fourth PC-to-PC ICP outputs a fourth transformation of the fourth PC-to-PC ICP and a fourth error value of the fourth PC-to-PC ICP.

2. The computerized method of claim 1 further comprising:

selecting a lowest error value ICP transformation of the first transformation, the second transformation, the third transformation, or the fourth transformation that has the lowest error value.

3. The computerized method of claim 1 , wherein a face tracking APIs comprises an ARKit API.

4. The computerized method of claim 1 , wherein the mask 1 comprises an ARKit facemask, and wherein the mask 2 comprises another ARKit facemask.

5. The computerized method of claim 1 further comprising:

outputting a lowest error value ICP transformation to an ARKit API.

6. The computerized method of claim 1 , wherein an ICP comprises an algorithm employed to minimize a difference between two 3D Face Point Clouds.

7. The computerized method of claim 6 , wherein the ICP is used to reconstruct 3D surfaces from a scan of a user's face.

8. The computerized method of claim 1 , wherein a first point cloud (PC)-to-mask ICP operation comprises registering the PC 1 to a corresponding face mask as mask 1 to correct a misalignment.

9. The computerized method of claim 1 , wherein a second point cloud (PC)-to-mask ICP operation comprises registering the PC 2 to a corresponding face mask as mask 2 to correct the misalignment.