IP Library Patent Application 19313816
Patent Application
App. No. 19/313,816

TECHNIQUES FOR NEUROMUSCULAR-SIGNAL-BASED DETECTION OF IN-AIR HAND GESTURES FOR TEXT PRODUCTION AND MODIFICATION, AND SYSTEMS, WEARABLE DEVICES, AND METHODS FOR USING THESE TECHNIQUES

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
19/313,816
Abstract

The various implementations described herein include methods and systems for producing and modifying text using neuromuscular-signal-sensing devices. In one aspect, a method includes causing the display of a plurality of text terms input by a user. Using data from one or more neuromuscular-signal sensors in communication with the wearable device, an in-air hand gesture performed by the user is detected while the text terms are displayed. In response to the in-air hand gesture, a text-modification mode is enabled that allows for modifying the text terms input by the user. A target term is identified and, while the text-modification mode is enabled, data about a voice input provided by the user for modifying the target term is received. The method further includes causing a modification to the target term in accordance with the voice input from the user.

Claims (61)

1 . An artificial-reality system, comprising:

one or more processors;

a display;

memory coupled to the one or more processors and the display, the memory storing instructions for:

while data associated with a messaging application is displayed on the display, detecting, using data from one or more neuromuscular-signal sensors, an in-air hand gesture performed by the user;

in response to the in-air hand gesture and in accordance with a determination that the in-air hand gesture corresponds to a microphone command, enabling a microphone to capture audio for use in conjunction with the messaging application;

receiving data about a voice input from the user via the microphone;

converting the voice input to a plurality of text terms; and

causing display, within the messaging application on the display, of the plurality of text terms to the user.

2 . The artificial-reality system of claim 1 , wherein the in-air gesture comprises a gesture during which the user's thumb is held against a user's digit for at least a predetermined period of time, and the memory further stores instructions for disabling the microphone in response to detecting release of the in-air hand gesture.

3 . The artificial-reality system of claim 1 , wherein the in-air hand gesture comprises a toggle gesture that is detected at a first point in time, and the memory further stores instructions for disabling the microphone in response a subsequent detection of the in-air hand gesture at a second point in time that is after the first point in time.

4 . The artificial-reality system of claim 1 , wherein the memory further stores instructions for:

while causing display of the plurality of text terms within the messaging application, detecting, using data from the one or more neuromuscular-signal sensors, an additional in-air hand gesture;

in response to the additional in-air gesture, enabling a text-modification mode;

while the text-modification mode is enabled, tracking a gaze of the user with respect to the plurality of text terms displayed within the messaging application; and

causing a term of the plurality of text terms to be emphasized in accordance with the tracking of the gaze of the user.

5 . The artificial-reality system of claim 1 , wherein the memory further stores instructions for:

after causing display of the plurality of text terms to the user, detecting, using data from the one or more neuromuscular-signal sensors, an additional in-air hand gesture, the additional in-air hand gesture being distinct from the in-air hand gesture;

in response to detecting the additional in-air hand gesture, causing the plurality of text terms to be sent to one or more other users via the messaging application.

6 . A non-transitory computer-readable storage medium storing one or more programs configured for execution by a wearable device having one or more processors and memory, the one or more programs comprising instructions for:

while data associated with a messaging application is displayed on a display that is in communication with a wearable device, detecting, using data from one or more neuromuscular-signal sensors in communication with the wearable device, an in-air hand gesture performed by the user;

in response to the in-air hand gesture and in accordance with a determination that the in-air hand gesture corresponds to a microphone command, enabling a microphone to capture audio for use in conjunction with the messaging application;

receiving data about a voice input from the user via the microphone;

converting the voice input to a plurality of text terms; and

causing display, within the messaging application on the display, of the plurality of text terms to the user.

7 . The non-transitory computer-readable storage medium of claim 6 , wherein the in-air gesture comprises a gesture during which the user's thumb is held against a user's digit for at least a predetermined period of time, and the one or more programs further comprise instructions for disabling the microphone in response to detecting release of the in-air hand gesture.

8 . The non-transitory computer-readable storage medium of claim 6 , wherein the in-air hand gesture comprises a toggle gesture that is detected at a first point in time, and the one or more programs further comprise instructions for disabling the microphone in response a subsequent detection of the in-air hand gesture at a second point in time that is after the first point in time.

9 . The non-transitory computer-readable storage medium of claim 6 , wherein the one or more programs further comprise instructions for:

while causing display of the plurality of text terms within the messaging application, detecting, using data from the one or more neuromuscular-signal sensors, an additional in-air hand gesture;

in response to the additional in-air gesture, enabling a text-modification mode;

while the text-modification mode is enabled, tracking a gaze of the user with respect to the plurality of text terms displayed within the messaging application; and

causing a term of the plurality of text terms to be emphasized in accordance with the tracking of the gaze of the user.

10 . The non-transitory computer-readable storage medium of claim 6 , wherein the one or more programs further comprise instructions for:

after causing display of the plurality of text terms to the user, detecting, using data from the one or more neuromuscular-signal sensors, an additional in-air hand gesture, the additional in-air hand gesture being distinct from the in-air hand gesture;

in response to detecting the additional in-air hand gesture, causing the plurality of text terms to be sent to one or more other users via the messaging application.

11 . A method performed at a wearable device having memory and one or more processors, the method comprising:

while data associated with a messaging application is displayed on a display that is in communication with a wearable device, detecting, using data from one or more neuromuscular-signal sensors in communication with the wearable device, an in-air hand gesture performed by the user;

in response to the in-air hand gesture and in accordance with a determination that the in-air hand gesture corresponds to a microphone command, enabling a microphone to capture audio for use in conjunction with the messaging application;

receiving data about a voice input from the user via the microphone;

converting the voice input to a plurality of text terms; and

causing display, within the messaging application on the display, of the plurality of text terms to the user.

12 . The method of claim 11 , wherein the in-air gesture comprises a gesture during which the user's thumb is held against a user's digit for at least a predetermined period of time, and the method further comprises disabling the microphone in response to detecting release of the in-air hand gesture.

13 . The method of claim 11 , wherein the in-air hand gesture comprises a toggle gesture that is detected at a first point in time, and the method further comprises disabling the microphone in response a subsequent detection of the in-air hand gesture at a second point in time that is after the first point in time.

14 . The method of claim 11 , further comprising:

while causing display of the plurality of text terms within the messaging application, detecting, using data from the one or more neuromuscular-signal sensors, an additional in-air hand gesture;

in response to the additional in-air gesture, enabling a text-modification mode;

while the text-modification mode is enabled, tracking a gaze of the user with respect to the plurality of text terms displayed within the messaging application; and

causing a term of the plurality of text terms to be emphasized in accordance with the tracking of the gaze of the user.

15 . The method of claim 14 , further comprising, while the text-modification mode is enabled:

detecting, using data from the one or more neuromuscular-signal sensors, one more in-air hand gesture performed by the user; and

in response to the one more in-air hand gesture, causing a deletion of the emphasized term.

16 . The method of claim 14 , wherein tracking of the gaze of the user is enabled in conjunction with enabling the text-modification mode.

17 . The method of claim 11 , further comprising:

in response to the in-air hand gesture, enabling an input mode;

while the input mode is enabled and after converting the voice input to the plurality of text terms, detecting, using data from the one or more neuromuscular-signal sensors, an additional hand gesture performed by the user; and

in response to the additional in-air hand gesture, causing a deletion of a term of the plurality of text terms.

18 . The method of claim 11 , further comprising:

after causing display of the plurality of text terms to the user, detecting, using data from the one or more neuromuscular-signal sensors, an additional in-air hand gesture, the additional in-air hand gesture being distinct from the in-air hand gesture;

in response to detecting the additional in-air hand gesture, causing the plurality of text terms to be sent to one or more other users via the messaging application.

19 . The method of claim 11 , wherein the wearable device is a wrist-wearable device that is configured to send instructions to a head-worn wearable device that includes the display.

20 . The method of claim 11 , wherein the wearable device is a head-mounted device that is configured to communicate with one or more additional wearable devices.