Granted Patent B2

US 12,300,219 · App. 18/474,853 · Granted May 13, 2025

Inventors: Sudharsan Krishnaswamy (San Jose, CA); Maisy Wieman (Boulder, CO); Jonah Probell (Alviso, CA)

Assignee: SoundHound AI IP, LLC

G10L15/063G10L13/02G10L15/16G10L15/1815G10L15/187G10L15/197G10L15/22

Abstract

Claims (53)

1. A method comprising: obtaining audio;

inferring, from the audio, a plurality of intent probabilities;

inferring, from the audio, a plurality of entity value probabilities;

in response to an intent probability exceeding an intent threshold, invoking a virtual assistant action, wherein the virtual assistant action is conditional based on a variable value probability exceeding a variable value threshold;

wherein the virtual assistant action is further conditional based on t he variable value probability exceeding the variable value threshold within a specific time period of the intent probability exceeding the intent threshold; and

passing an entity value with the highest probability as an argument for the virtual assistant action.

2. The method of claim 1 wherein the inferring of intent probabilities uses a model trained to recognize speech in a plurality of human languages.

3. The method of claim 1 wherein invoking the virtual assistant action is conditional based on having not previously invoked an action within a specific amount of time.

4. The method of claim 1 wherein invoking the virtual assistant action is conditional based on end-of-utterance detection on the audio.

5. The method of claim 1 further comprising:

directly inferring, from the audio, a plurality of variable value probabilities,

wherein the virtual assistant action comprises an argument indicating which variable value probability is the highest.

6. The method of claim 1 further comprising:

directly inferring, from the audio, a domain probability,

wherein the virtual assistant action is conditional based on the domain probability exceeding a domain threshold.

7. The method of claim 1 further comprising:

directly inferring, from the audio, a domain probability,

wherein the virtual assistant action is conditional based on the domain probability exceeding a domain threshold;

directly inferring, from the audio, a plurality of variable value probabilities; and

the virtual assistant action comprising an argument indicating which variable value probability is the highest.

8. A method comprising:

obtaining audio;

inferring, from the audio:

(a) a domain probability;

(b) a plurality of intent probabilities; and

(c) a plurality of variable value probabilities; and

in response to:

(A) the domain probability exceeding a domain threshold;

(B) an intent probability exceeding an intent threshold when the audio includes speech of words in one of a plurality of recognized natural languages;

(C) a variable value probability exceeding a variable value threshold within a certain period of time of the intent probability exceeding the intent threshold;

(D) an end of an utterance detection signal; and

(E) a specific amount of time having elapsed,

invoking a virtual assistant action comprising an argument indicating which variable value probability is the highest.

9. A device comprising:

a microphone;

a speaker;

one or more computer processors; and

a memory storing code that causes the one or more computer processors to:

obtain audio from the microphone;

infer, from the audio, a plurality of intent probabilities;

infer, from the audio, a plurality of entity value probabilities;

in response to an intent probability exceeding an intent threshold, retrieve information, wherein retrieving the information is conditional based on which intent probability is the highest;

wherein the retrieving information is conditional based on having at least one entity value probability exceeding an entity value threshold;

wherein retrieving information is further conditional based on the entity value probability exceeding the entity value threshold within a specific time period of the intent probability exceeding the intent threshold;

pass an entity value with the highest probability as an argument for the information retrieval;

in response to retrieving the information, synthesize speed audio using text-to-speech; and

output the synthesized speech audio to the speaker.

10. The device of claim 9 wherein the inferring of intent probabilities uses a model trained to recognize speech in a plurality of human languages.

11. The device of claim 9 wherein retrieving information is conditional based on having not previously retrieved information within a specific amount of time.

12. The device of claim 9 wherein retrieving information is conditional based on end-of-utterance detection on the audio.

13. The device of claim 9 further comprising:

inferring, without transcription, from the audio, a domain probability,

wherein retrieving informamtion is conditional based on the domain probability exceeding a domain threshold.

Assignments (3)

Continuity (3)

References Cited (64)