Granted Patent B2

US 12,217,536 · App. 17/766,688 · Granted Feb 4, 2025

Inventors: Kevin Chyn (San Jose, CA); James Brooks Miller (Sunnyvale, CA); Tyler Reed Kugler (Palo Alto, CA)

Assignee: Google LLC

G06V40/161G06V10/143G06V40/165G06V40/166G06V40/50G06V40/67G06V10/17

Abstract

Claims (69)

1. A method for a grid-based enrollment for face authentication by a user device, the method comprising:

responsive to a user input, presenting a preview image, captured by a camera, via a display device, the presenting to initiate enrollment for face authentication;

overlaying a two-dimensional (2D) object over the preview image, the 2D object having a region indicating an approximate orientation for a user to position their face relative to the camera;

responsive to a determination that the user's face is positioned at the approximate orientation, removing the 2D object and presenting a three-dimensional (3D ) tracking window as an overlay over the preview image, the 3D tracking window having a plurality of segments;

tracking an approximate direction that the user's face is facing relative to the camera;

based on the tracking:

highlighting one or more segments of the plurality of segments of the 3D tracking window that correspond to the approximate direction that the user's face is facing;

capturing one or more enrollment images of the user's face facing the approximate direction;

generating one or more embeddings based on the one or more enrollment images;

storing, in a secure storage unit, the one or more embeddings in a fixed grid of pose cells corresponding to various facial poses for use in face authentication; and

responsive to generation and storage of the one or more embeddings, providing an indication that the one or more segments are complete.

2. The method of claim 1 , wherein the providing of the indication that the one or more segments are complete comprises changing an opacity of the one or more segments to indicate progress.

3. The method of claim 1 , further comprising, responsive to completing the enrollment:

presenting a confirmation of completion of the enrollment; and

enabling the one or more embeddings stored in the secure storage unit to be used to unlock the user device via the face authentication.

4. The method of claim 1 , wherein the 3D tracking window is shaped to direct the user to roll or rotate their head.

5. The method of claim 1 , wherein the 3D tracking window comprises a hemisphere.

6. The method of any preceding claim 1 , wherein the preview image is a live preview.

7. The method of claim 1 , further comprising:

responsive to presenting the 3D tracking window, initiating a timer;

resetting the timer upon completion of a respective segment of the plurality of segments; and

responsive to the timer expiring without one or more new segments of the plurality of segments being completed, providing an indication to direct the user to face a new direction corresponding to at least one of the one or more new segments.

8. The method of claim 1 , further comprising:

determining that a first segment, of the plurality of segments, near an edge of the 3D tracking window is incomplete and multiple segments adjacent to the first segment are complete; and

indicating to the user that the first segment is complete without capturing an enrollment image of the user's face facing a direction corresponding to the first segment.

9. The method of claim 1 , wherein the plurality of segments each correspond to a pose of their head.

10. The method of claim 1 , wherein the pose cells in the grid of pose cells are characterized by pan angle and tilt angle.

11. The method of claim 1 , wherein:

the highlighting of the one or more segments comprises:

fading in a first segment corresponding to the approximate direction that the user's face is facing relative to the camera; and

fading out the first segment when the approximate direction that the user's face is facing changes and no longer corresponds to the first segment; and

a first animation speed corresponding to the fading in is greater than a second animation speed corresponding to the fading out.

12. The method of claim 1 , wherein the indication that the one or more segments are complete comprises haptic feedback.

13. The method of claim 1 , wherein the indication that the one or more segments are complete comprises audible feedback.

14. The method of claim 1 , wherein:

the camera comprises one or more color cameras and one or more near-infrared cameras;

the one or more color cameras are used to capture the preview image; and

the capturing of the one or more enrollment images of the user's face comprises capturing, using the one or more near-infrared cameras, one or more near-infrared images usable to generate the one or more embeddings.

15. A user device comprising:

a camera system configured to capture images of a face of a user for face authentication;

a display device configured to display a for displaying the preview image including the user's face; and

a processor and memory configured to implement an enrollment module configured to:

responsive to initiation of an enrollment process, cause a two-dimensional (2D ) object to be overlaid over the preview image, the 2D object comprising a center region indicating an approximate orientation for the user to position their face relative to the camera system;

responsive to a determination that the user's face is positioned within the center region, present a three-dimensional (3D ) tracking window as an overlay over the preview image, the 3D tracking window having a plurality of segments, the plurality of segments persisting to correspond to an approximate direction that the user's face is facing relative to the camera system;

track the approximate direction that the user's face is facing;

determine one or more segments of the plurality of segments that correspond to the approximate direction that the user's face is facing;

highlight the one or more segments to provide visual feedback to the user;

cause the camera system to capture one or more enrollment images corresponding to a pose of the user's face facing the approximate direction;

generate one or more embeddings corresponding to the one or more enrollment images; and

responsive to an indication that the one or more embeddings have been generated, provide an indication that the one or more segments are complete.

16. The user device of claim 15 , wherein the 3D tracking window is shaped as a hemisphere and is overlaid over the preview image such that the user's face in the preview image is displayed within the hemisphere.

17. The user device of claim 15 , wherein the enrollment module is further configured to:

responsive to presentation of the 3D tracking window, initiate a timer;

reset the timer upon completion of a respective segment of the plurality of segments; and

responsive to the timer expiring without one or more new segments of the plurality of segments being completed, provide an indication to direct the user to face a direction of at least one of the one or more new segments.

18. The user device of claim 15 , wherein the plurality of segments each correspond to a pose of the user's head.

19. A computer-readable storage media comprising instructions that, when executed, configure at least one processor of a user device to:

display a preview image, captured by a camera system of the user device, via a display device during an enrollment process for face authentication;

cause a two-dimensional (2D ) object to be overlaid over the preview image, the 2D object comprising a center region indicating an approximate orientation for a user to position their face relative to the camera system;

responsive to a determination that the user's face is positioned within the center region, present a three-dimensional (3D ) tracking window as an overlay over the preview image, the 3D tracking window having a plurality of segments, the plurality of segments persisting to correspond to an approximate direction that the user's face is facing relative to the camera system;

track the approximate direction that the user's face is facing;

identify one or more segments of the plurality of segments that correspond to the approximate direction that the user's face is facing;

highlight the one or more segments to provide visual feedback to the user;

cause the camera system to capture one or more enrollment images corresponding to a pose of the user's face facing the approximate direction;

generate embeddings corresponding to the one or more enrollment images; and

responsive to an indication that embeddings have been generated for the pose, provide an indication that the one or more segments are complete.

20. The computer-readable storage media of claim 19 , wherein:

the embeddings include a pair of 2D and 3D embeddings for a pose of the user's face facing the approximate direction; and

the at least one processor is configured to store the embeddings in a fixed grid of pose cells, which are characterized by pan angle and tilt angle.

Assignments (1)

Continuity (2)

References Cited (23)