Granted Patent B2

US 12,243,531 · App. 18/506,866 · Granted Mar 4, 2025

Inventors: Anthony Robert Sheeder (Fort Lauderdale, FL); Colby Nelson Leider (Coral Gables, FL)

Assignee: Magic Leap, Inc.

G10L15/22G06F3/013G10L15/14G10L15/25G10L15/30G10L2015/223G10L2015/227

Abstract

Claims (38)

1. A method of presenting a signal to a speech processing engine, the method comprising:

receiving, via a first microphone of a wearable head device, an audio signal comprising speech from a user of the wearable head device;

identifying a first portion of the audio signal;

determining, for the first portion of the audio signal, a first probability that the first portion comprises speech directed by a user as input to the speech processing engine;

identifying a second portion of the audio signal, the second portion having a start time in the audio signal and an end time in the audio signal, wherein the first portion comprises speech located before the start time in the audio signal and further comprises speech located after the end time in the audio signal;

determining, for the second portion of the audio signal, a second probability that the second portion comprises speech directed by the user as input to the speech processing engine; and

in accordance with a determination that the first probability exceeds a first threshold, and further in accordance with a determination that the second probability does not exceed the first threshold:

presenting the first portion of the audio signal as a first input to the speech processing engine, and

forgoing presenting the second portion of the audio signal as an input to the speech processing engine.

2. The method of claim 1 , wherein the first probability is determined based on a comparison of the first portion of the audio signal to a plurality of audio signals in a database, each audio signal of the plurality of audio signals associated with a probability that its respective audio signal comprises speech directed as input to the speech processing engine.

3. The method of claim 1 , further comprising receiving first sensor data from one or more sensors associated with the wearable head device, wherein the one or more sensors do not include the first microphone.

4. The method of claim 3 , wherein the first probability is determined further based on a comparison of the first sensor data to a plurality of sensor data in a database, a sensor data of the plurality of sensor data in the database associated with a respective audio signal and further associated with a probability that the respective audio signal comprises speech directed as input to the speech processing engine.

5. The method of claim 3 , wherein the first sensor data is indicative of one or more of a position, an orientation, an eye movement, an eye gaze target, or a vital sign of the user.

6. The method of claim 3 , further comprising determining based on the first sensor data whether the first portion of the audio signal corresponds to the user and in accordance with a determination that the audio signal does not correspond to the user, discarding the first portion of the audio signal.

7. The method of claim 1 , further comprising determining a query based on the first portion of the audio signal, wherein the first portion of the audio signal forms a portion of the query.

8. The method of claim 1 , wherein identifying the second portion of the audio signal comprises determining a boundary between the first portion of the audio signal and the second portion of the audio signal.

9. The method of claim 8 , wherein the boundary is determined based on one or more of: a period of silence between speech in the first portion and speech in the second portion, a change in pitch or intonation of the speech, a change in cadence of the speech, or a breathing pattern of the user.

10. The method of claim 8 , wherein the boundary is determined based on sensor data from one or more sensors associated with the wearable head device, wherein the one or more sensors do not include the first microphone.

11. A system for providing input to a speech processing engine, the system including:

a wearable head device comprising a microphone; and one or more processors configured to perform:

receiving, via the microphone, an audio signal comprising speech from a user of the wearable head device;

identifying a first portion of the audio signal;

determining, for the first portion of the audio signal, a first probability that the first portion comprises speech directed by a user as input to the speech processing engine;

identifying a second portion of the audio signal, the second portion having a start time in the audio signal and an end time in the audio signal, wherein the first portion comprises speech located before the start time in the audio signal and further comprises speech located after the end time in the audio signal;

determining, for the second portion of the audio signal, a second probability that the second portion comprises speech directed by the user as input to the speech processing engine; and

in accordance with a determination that the first probability exceeds a first threshold, and further in accordance with a determination that the second probability does not exceed the first threshold:

presenting the first portion of the audio signal as a first input to the speech processing engine, and

forgoing presenting the second portion of the audio signal as an input to the speech processing engine.

12. The system of claim 11 , wherein the first probability is determined further based on a comparison of the first portion of the audio signal to a plurality of audio signals in a database, a respective audio signal of the plurality of audio signals associated with a probability that the respective audio signal comprises speech directed as input to the speech processing engine.

13. The system of claim 11 , further comprising one or more sensors associated with the wearable head device, wherein the one or more processors are further configured to perform:

receiving first sensor data from the user via the one or more sensors, wherein the first probability is determined further based on a comparison of the first sensor data to a plurality of sensor data in a database, each sensor data of the plurality of sensor data in the database associated with an audio signal and further associated with a probability that its respective audio signal comprises speech directed as input to the speech processing engine.

14. The system of claim 13 , wherein the first sensor data is indicative of one or more of a position of the user, an orientation of the user, an eye movement of the user, an eye gaze target of the user, and a vital sign of the user.

15. The system of claim 11 , wherein the wearable head device includes the one or more processors.

16. The system of claim 11 , wherein the system includes a vehicle including the one or more processors.

17. The system of claim 11 , wherein the system includes an electronic voice assistant including the one or more processors.

18. The system of claim 11 , wherein identifying the second portion of the audio signal comprises determining a boundary between the first portion of the audio signal and the second portion of the audio signal.

19. The system of claim 18 , wherein the boundary is determined based on one or more of: a period of silence between speech in the first portion and speech in the second portion, a change in pitch or intonation of the speech, a change in cadence of the speech, or a breathing pattern of the user.

20. The system of claim 18 , wherein the boundary is determined based on sensor data from one or more sensors associated with the wearable head device, wherein the one or more sensors do not include the first microphone.

Assignments (3)

Continuity (4)

References Cited (237)

Cited By (5)

US 12,327,573 US 12,347,448 US 12,417,766 US 12,688,845 US 12,696,045