IP Library Granted Patent US 12,067,171
Granted Patent B1
US 12,067,171 · App. 17/737,483 · Granted Aug 20, 2024

Systems and methods for operating artificial reality devices using gestures

Inventor: Priyanka Sharma (Cupertino, CA)
Assignee: Meta Platforms, Inc.
G06F3/017G02B27/0172G06F3/011G06T19/006G09B21/00G09B21/001G09B21/009
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,067,171
App. No.
17/737,483
Granted
Aug 20, 2024
Kind
B1
Abstract

Systems and methods are provided for operating an artificial reality device using gestures. In various exemplary embodiments, an artificial reality device may receive information initiating a gesture recognition mode via the artificial reality device, and cause a camera associated with the artificial reality device to operate in the gesture recognition mode. The artificial reality device may further receive at least one image of a gesture captured by the camera and identify and evaluate the gesture. The gesture may be indicative of a user command. The artificial reality device may also execute the user command, and may generate feedback confirming execution of the user command.

Claims (51)

1. An artificial reality device comprising at least one processor and a non-transitory memory including computer-executable instructions which, when executed by the processor, cause the artificial reality device to:

determine, based on a first user interaction with the artificial reality device, a type of impairment comprising at least one of a visual impairment or an auditory impairment;

initiate a gesture recognition mode, based on the type of impairment, to evaluate a second user interaction with the artificial reality device;

cause a camera associated with the artificial reality device to operate in the gesture recognition mode;

receive at least one image of a gesture captured by the camera;

identify and evaluate the gesture, wherein the gesture is indicative of a user command;

execute the user command in response to the evaluated gesture; and

generate feedback confirming execution of the user command.

2. The artificial reality device of claim 1 , wherein the instructions, when executed by the processor, further cause the artificial reality device to:

initiate the gesture recognition mode via at least one of a user operating the artificial reality device, a physical selection, an auditory command, or a lapse of a period of time.

3. The artificial reality device of claim 1 , wherein the instructions, when executed by the processor, further cause the artificial reality device to:

provide the generated feedback via at least one of an audio output, a visual output, or a sensory output.

4. The artificial reality device of claim 1 , wherein the instructions, when executed by the processor, further cause the artificial reality device to:

display the generated feedback via a user interface.

5. The artificial reality device of claim 1 , wherein the instructions, when executed by the processor, further cause the artificial reality device to:

evaluate the gesture via a trained machine learning model comprising one or more trained gestures.

6. The artificial reality device of claim 1 , wherein the user command comprises at least one of a selection, a rejection, or an interaction with the artificial reality device.

7. The artificial reality device of claim 1 , wherein the artificial reality device comprises at least one of artificial reality glasses or an artificial reality headset.

8. The artificial reality device of claim 1 , wherein the gesture comprises one or more of a thumbs up, a thumbs down, a directional indication, a swiping motion, a pinching motion, or an expansion motion.

9. A method for interacting with an artificial reality device comprising:

determining, based on a first user interaction with the artificial reality device, a type of impairment comprising at least one of a visual impairment or an auditory impairment;

initiating a gesture recognition mode, based on the type of impairment, to evaluate a second user interaction with the artificial reality device;

causing, via the artificial reality device, a camera to operate in the gesture recognition mode;

receiving at least one image of a gesture captured by the camera associated with the artificial reality device;

identifying and evaluating the gesture;

executing a user command in response to the evaluated gesture; and

generating feedback, via the artificial reality device, in response to execution of the user command.

10. The method of claim 9 , further comprising:

initiating the gesture recognition mode via at least one of a user operating the artificial reality device, a physical selection, an auditory command, or a lapse of a period of time.

11. The method of claim 9 , further comprising:

providing the generated feedback via at least one of an audio output, a visual output, or a sensory output.

12. The method of claim 9 , further comprising:

displaying the generated feedback via a user interface.

13. The method of claim 9 , further comprising:

evaluating the gesture via a trained machine learning model comprising one or more trained gestures.

14. The method of claim 9 , wherein the user command comprises at least one of a selection, a rejection, or an interaction with the artificial reality device.

15. The method of claim 9 , wherein the artificial reality device comprises at least one of artificial reality glasses or an artificial reality headset.

16. The method of claim 9 , wherein the gesture comprises one or more of a thumbs up, a thumbs down, a directional indication, a swiping motion, a pinching motion, or an expansion motion.

17. A non-transitory computer-readable storage medium storing instructions that, when executed, cause:

determining, based on a first user interaction with an artificial reality device, a type of impairment comprising at least one of a visual impairment or an auditory impairment;

initiating a gesture recognition mode, based on the type of impairment, to evaluate a second user interaction with the artificial reality device;

facilitating, via the artificial reality device, a camera to operate in the gesture recognition mode;

receiving at least one image of a gesture captured by the camera associated with the artificial reality device;

identifying and evaluating the gesture;

executing a user command in response to the evaluated gesture; and

generating feedback, via the artificial reality device, in response to execution of the user command.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the instructions, when executed, further cause:

initiating the gesture recognition mode via at least one of a user operating the artificial reality device, a physical selection, an auditory command, or a lapse of a period of time.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the instructions, when executed, further cause:

evaluating the gesture via a trained machine learning model comprising one or more trained gestures.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the gesture comprises one or more of a thumbs up, a thumbs down, a directional indication, a swiping motion, a pinching motion, or an expansion motion.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2022
From: SHARMA, PRIYANKA
To: META PLATFORMS, INC.
Reel/Frame 059842/0680 →
Continuity (1)
Provisional Application 63231908 · Aug 11, 2021