Granted Patent B1

US 12,567,404 · App. 18/050,041 · Granted Mar 3, 2026

Inventors: Michael Greenberg (Oakland, CA); Aaron Jackson (Snohomish, WA)

Assignee: Meta Platforms, Inc.

G10L15/1815G10L15/187G10L15/22G10L2015/223G10L15/30

Abstract

Claims (60)

1 . A method comprising, by one or more computing systems:

receiving, from an extended reality (XR) display device, an audio input of a user of the XR display device, wherein the audio input comprises a first voice command of a plurality of voice commands associated with a first application, wherein the plurality of voice commands are executable by a natural-language understanding (NLU) model;

determining a first context of the user with respect to an extended reality (XR) environment;

determining, based on the first context, a tunable confidence interval for the NLU model to match the audio input to the first voice command, wherein the tunable confidence interval varies based on user context, and wherein the tunable confidence interval is set to a first tunable confidence level based on the first context of the user;

determining, by the NLU model, whether the audio input matches the first voice command based on the first tunable confidence level; and

executing, by the first application, a first task corresponding to the first voice command responsive to determining whether the audio input matches the first voice command.

2 . The method of claim 1 , wherein the plurality of voice commands comprise one or more of avatar movement, avatar actions, object movement, object actions, application settings, or system settings.

3 . The method of claim 1 , further comprising:

adjusting the tunable confidence interval to a second tunable confidence level based on content of the first application, wherein the second tunable confidence level is higher than the first tunable confidence level, and wherein determining the audio input matches the first voice command is further based on the second tunable confidence level.

4 . The method of claim 1 , further comprising:

adjusting the tunable confidence interval to a second tunable confidence level, wherein the second tunable confidence level is different from the first tunable confidence level; and

determining, by a customizable NLU model of the XR display device, whether the audio input matches the first voice command based on the second tunable confidence level.

5 . The method of claim 1 , further comprising:

providing instructions to the XR display device to place one or more microphones of the XR display device into a listening mode responsive determining the first tunable confidence level exceeds a threshold tunable confidence level.

6 . The method of claim 1 , further comprising:

determining the audio input comprises one or more words that are phonetically adjacent to one or more other words; and

adjusting the tunable confidence interval to a second tunable confidence level responsive to determining the audio input comprises one or more words that are phonetically adjacent to one or more other words, wherein the second tunable confidence level is higher than the first tunable confidence level, and wherein determining the audio input matches the first voice command is further based on the second tunable confidence level.

7 . The method of claim 1 , further comprising:

determining a difficulty associated with the first task; and

adjusting the tunable confidence interval to a second tunable confidence level based on the difficulty associated with the first task, wherein determining the audio input matches the first voice command is further based on the second tunable confidence level.

8 . The method of claim 1 , further comprising:

accessing a list of predetermined tasks; and

adjusting the tunable confidence interval to a second tunable confidence level based on the first task matching one task out of the list of predetermined tasks, wherein determining the audio input matches the first voice command is further based on the second tunable confidence level.

9 . The method of claim 1 , further comprising:

receiving an indication that the first user has a speech characteristic; and

adjusting the tunable confidence interval to a second tunable confidence level based on the speech characteristic, wherein determining the audio input matches the first voice command is further based on the second tunable confidence level.

10 . The method of claim 1 , wherein determining whether the audio input matches the first voice command further comprises:

identifying one or more intents or one or more slots associated with the audio input;

accessing one or more first voice command intents or one or more first voice command slots associated with the first voice command; and

determining whether the identified one or more intents or the one or more slots match the one or more first voice command intents or the one or more first voice command slots based on the first tunable confidence level.

11 . One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive, from an extended reality (XR) display device, an audio input of a user of the XR display device, wherein the audio input comprises a first voice command of a plurality of voice commands associated with a first application, wherein the plurality of voice commands are executable by a natural-language understanding (NLU) model;

determine a first context of the user with respect to an extended reality (XR) environment;

determine, based on the first context, a tunable confidence interval for the NLU model to match the audio input to the first voice command, wherein the tunable confidence interval varies based on user context, and wherein the tunable confidence interval is set to a first tunable confidence level based on the first context of the user;

determine, by the NLU model, whether the audio input matches the first voice command based on the first tunable confidence level; and

execute, by the first application, a first task corresponding to the first voice command responsive to determining whether the audio input matches the first voice command.

12 . The media of claim 11 , wherein the plurality of voice commands comprise one or more of avatar movement, avatar actions, object movement, object actions, application settings, or system settings.

13 . The media of claim 11 , wherein the software is further operable when executed to:

adjust the tunable confidence interval to a second tunable confidence level based on content of the first application, wherein the second tunable confidence level is higher than the first tunable confidence level, and wherein determining the audio input matches the first voice command is further based on the second tunable confidence level.

14 . The media of claim 11 , wherein the software is further operable when executed to:

adjust the tunable confidence interval to a second tunable confidence level, wherein the second tunable confidence level is different from the first tunable confidence level; and

determine, by a customizable NLU model of the XR display device, whether the audio input matches the first voice command based on the second tunable confidence level.

15 . The media of claim 11 , wherein the software is further operable when executed to:

provide instructions to the XR display device to place one or more microphones of the XR display device into a listening mode responsive determining the first tunable confidence level exceeds a threshold tunable confidence level.

16 . A system comprising:

one or more processors; and

a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:

receive, from an extended reality (XR) display device, an audio input of a user of the XR display device, wherein the audio input comprises a first voice command of a plurality of voice commands associated with a first application, wherein the plurality of voice commands are executable by a natural-language understanding (NLU) model;

determine a first context of the user with respect to an extended reality (XR) environment;

determine, based on the first context, a tunable confidence interval for the NLU model to match the audio input to the first voice command, wherein the tunable confidence interval varies based on user context, and wherein the tunable confidence interval is set to a first tunable confidence level based on the first context of the user;

determine, by the NLU model, whether the audio input matches the first voice command based on the first tunable confidence level; and

execute, by the first application, a first task corresponding to the first voice command responsive to determining whether the audio input matches the first voice command.

17 . The system of claim 16 , wherein the plurality of voice commands comprise one or more of avatar movement, avatar actions, object movement, object actions, application settings, or system settings.

18 . The system of claim 16 , wherein the processors are further operable when executing the instructions to:

adjust the tunable confidence interval to a second tunable confidence level based on content of the first application, wherein the second tunable confidence level is higher than the first tunable confidence level, and wherein determining the audio input matches the first voice command is further based on the second tunable confidence level.

19 . The system of claim 16 , wherein the processors are further operable when executing the instructions to:

adjust the tunable confidence interval to a second tunable confidence level, wherein the second tunable confidence level is different from the first tunable confidence level; and

determine, by a customizable NLU model of the XR display device, whether the audio input matches the first voice command based on the second tunable confidence level.

20 . The system of claim 16 , wherein the processors are further operable when executing the instructions to:

provide instructions to the XR display device to place one or more microphones of the XR display device into a listening mode responsive determining the first tunable confidence level exceeds a threshold tunable confidence level.

Assignments (1)

Continuity (1)

References Cited (347)