Granted Patent B2

US 12,360,663 · App. 17/729,808 · Granted Jul 15, 2025

Inventors: Sharon Moll (Lachen, CH); Dawei Zhang (Seattle, WA)

Assignee: Snap Inc.

G06F3/04886G06F3/011G06F3/017G06F3/0426

Abstract

Claims (49)

1. A computer-implemented method comprising, by one or more processors of an Augmented Reality (AR) system:

detecting, using one or more cameras of the AR system, a start/stop text entry gesture made by a user of the AR system;

generating, in response to detecting the start/stop text entry gesture, a virtual keyboard user interface including a virtual keyboard having a plurality of virtual keys;

providing to the user, the virtual keyboard user interface using a display of the AR system;

continuously performing operations comprising:

in response to detecting, using the one or more cameras of the AR system, a hold of an enter text gesture made by the user, the enter text gesture made by the user using a free hand of the user,

generating entered text data including a current word by collecting, using the one or more cameras of the AR system, continuous motion gesture data of a continuous motion as the user makes the continuous motion through the virtual keys of the virtual keyboard, the continuous motion made by the user using a text entry hand of the user, the text entry hand different than the free hand of the user;

in response to detecting, using the one or more cameras of the AR system, an end word gesture by the free hand of the user, ending the current word and inserting a word delimiter into the entered text data; and

in response to detecting, using the one or more cameras of the AR system, the user making the start/stop text entry gesture, closing the virtual keyboard user interface.

2. The computer-implemented method of claim 1 , wherein generating the entered text data based on the collected continuous motion gesture data further comprises:

mapping the collected continuous motion gesture data to text data based on a continuous motion gesture model generated using machine learning; and

generating the entered text data based on the text data.

3. The computer-implemented method of claim 1 , further comprising providing the entered text data to the user using a display of the AR system.

4. The computer-implemented method of claim 1 , wherein the AR system comprises a head-worn device.

5. A computing apparatus of an AR system, comprising:

one or more processors; and

a memory storing instructions that, when executed by the one or more processors, cause the computing apparatus to perform operations comprising:

detecting, using one or more cameras of the AR system, a start/stop text entry gesture made by a user of the AR system;

generating, in response to detecting the start/stop text entry gesture, a virtual keyboard user interface including a virtual keyboard having a plurality of virtual keys;

providing to the user the virtual keyboard user interface using a display of the AR system;

continuously performing operations comprising:

in response to detecting, using the one or more cameras of the AR system, a hold of an enter text gesture made by the user, the enter text gesture made by the user using a free hand of the user,

generating entered text data including a current word by collecting, using the one or more cameras of the AR system, continuous motion gesture data of a continuous motion as the user makes the continuous motion through the virtual keys of the virtual keyboard, the continuous motion made by the user using a text entry hand of the user, the text entry hand different than the free hand of the user;

in response to detecting, using the one or more cameras of the AR system, an end word gesture by the free hand of the user, ending the current word and inserting a word delimiter into the entered text data; and

in response to detecting, using the one or more cameras of the AR system, the user making the start/stop text entry gesture, closing the virtual keyboard user interface.

6. The computing apparatus of claim 5 , wherein the instructions further cause the computing apparatus to perform operations comprising:

detecting using the one or more cameras of the AR system, an end text entry gesture made by the user; and

closing, in response to detecting the end text entry gesture, the virtual keyboard user interface.

7. The computing apparatus of claim 5 , wherein the instructions further cause the computing apparatus to perform operations comprising:

providing the entered text data to the user using a display of the AR system.

8. The computing apparatus of claim 5 , wherein the AR system comprises a head-worn device.

9. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:

detecting, using one or more cameras of an AR system, a start/stop text entry gesture made by a user of the AR system;

generating, in response to detecting the start/stop text entry gesture, a virtual keyboard user interface including a virtual keyboard having a plurality of virtual keys;

providing to the user, the virtual keyboard user interface using a display of the AR system;

continuously performing operations comprising:

in response to detecting, using the one or more cameras of the AR system, a hold of an enter text gesture made by the user, the enter text gesture made by the user using a free hand of the user,

generating entered text data including a current word by collecting, using the one or more cameras of the AR system, continuous motion gesture data of a continuous motion as the user makes the continuous motion through the virtual keys of the virtual keyboard, the continuous motion made by the user using a text entry hand of the user, the text entry hand different than the free hand of the user;

in response to detecting, using the one or more cameras of the AR system, an end word gesture by the free hand of the user, ending the current word and inserting a word delimiter into the entered text data; and

in response to detecting, using the one or more cameras of the AR system, the user making the start/stop text entry gesture, closing the virtual keyboard user interface.

10. The non-transitory computer-readable storage medium of claim 9 , wherein the instructions further cause the computer to perform operations comprising:

detecting using the one or more cameras of the AR system, an end text entry gesture made by the user; and

closing the virtual keyboard user interface.

11. The non-transitory computer-readable storage medium of claim 9 , wherein the instructions causing the computer to perform the operation of generating the entered text data further cause the computer to perform operations comprising:

mapping the collected continuous motion gesture data to text data based on a continuous motion gesture model generated using machine learning; and

generating the entered text data based on the text data.

12. The non-transitory computer-readable storage medium of claim 9 , wherein the instructions further cause the computer to perform operations comprising:

providing the entered text data to the user using a display of the AR system.

13. The non-transitory computer-readable storage medium of claim 9 , wherein the AR system comprises a head-worn device.

Assignments (1)

Continuity (1)

References Cited (237)