Granted Patent B2

US 11,380,310 · App. 16/998,786 · Granted Jul 5, 2022

Inventors: Alejandro Acero (Monte Sereno, CA); Hepeng Zhang (Sunnyvale, CA)

Assignee: Apple Inc.

G10L15/1822G06F3/16G10L13/04G10L15/183G10L15/22G10L15/30G10L25/78G10L25/87G10L2015/223

Abstract

Claims (83)

1. An electronic device, comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving a user utterance;

determining, based on a plurality of candidate text representations of the user utterance, a plurality of candidate user intents for the user utterance, wherein each candidate user intent of the plurality of candidate user intents corresponds to a respective candidate task flow of a plurality of candidate task flows;

determining a plurality of task flow scores for the plurality of candidate task flows, each task flow score of the plurality of task flow scores corresponding to a respective candidate task flow of the plurality of candidate task flows;

selecting, based on the plurality of task flow scores, a first candidate task flow of the plurality of candidate task flows; and

executing the first candidate task flow, including presenting, to the user, results from executing the first candidate task flow.

2. The electronic device of claim 1 , wherein the one or more programs include further instructions for:

for each candidate task flow of the plurality of candidate task flows:

resolving one or more flow parameters of the respective candidate task flow, wherein a respective task flow score for the respective candidate task flow is based on resolving the one or more flow parameters of the respective candidate task flow.

3. The electronic device of claim 2 , wherein resolving the one or more flow parameters of the respective candidate task flow comprises searching a data source for one or more values corresponding to the one or more flow parameters, the data source corresponding to one or more properties of a respective candidate user intent of the plurality of candidate user intents.

4. The electronic device of claim 1 , wherein:

each candidate text representation of the plurality of candidate text representations has an associated speech recognition confidence score; and

each task flow score of the plurality of task flow scores is based on a respective speech recognition confidence score of a respective candidate text representation of the plurality of candidate text representations.

5. The electronic device of claim 1 , wherein:

each candidate user intent of the plurality of candidate user intents has an associated intent confidence score; and

each task flow score of the plurality of task flow scores is based on a respective intent confidence score of a respective candidate user intent of the plurality of candidate user intents.

6. The electronic device of claim 1 , wherein:

a first candidate user intent of the plurality of candidate user intents is determined from a first candidate text representation of the plurality of candidate text representations; and

a second candidate user intent of the plurality of candidate user intents is determined from a second candidate text representation of the plurality of candidate text representations.

7. The electronic device of claim 1 , wherein the one or more programs include further instructions for:

ranking the plurality of candidate task flows in accordance with the plurality of task flow scores, wherein selecting the first candidate task flow is based on the ranking of the plurality of candidate task flows.

8. A method for operating a digital assistant, the method comprising:

at an electronic device having one or more processors and memory:

receiving a user utterance;

determining, based on a plurality of candidate text representations of the user utterance, a plurality of candidate user intents for the user utterance, wherein each candidate user intent of the plurality of candidate user intents corresponds to a respective candidate task flow of a plurality of candidate task flows;

determining a plurality of task flow scores for the plurality of candidate task flows, each task flow score of the plurality of task flow scores corresponding to a respective candidate task flow of the plurality of candidate task flows;

selecting, based on the plurality of task flow scores, a first candidate task flow of the plurality of candidate task flows; and

executing the first candidate task flow, including presenting, to the user, results from executing the first candidate task flow.

9. The method of claim 8 , further comprising:

for each candidate task flow of the plurality of candidate task flows:

resolving one or more flow parameters of the respective candidate task flow, wherein a respective task flow score for the respective candidate task flow is based on resolving the one or more flow parameters of the respective candidate task flow.

10. The method of claim 9 , wherein resolving the one or more flow parameters of the respective candidate task flow comprises searching a data source for one or more values corresponding to the one or more flow parameters, the data source corresponding to one or more properties of a respective candidate user intent of the plurality of candidate user intents.

11. The method of claim 8 , wherein:

each candidate text representation of the plurality of candidate text representations has an associated speech recognition confidence score; and

each task flow score of the plurality of task flow scores is based on a respective speech recognition confidence score of a respective candidate text representation of the plurality of candidate text representations.

12. The method of claim 8 , wherein:

each candidate user intent of the plurality of candidate user intents has an associated intent confidence score; and

each task flow score of the plurality of task flow scores is based on a respective intent confidence score of a respective candidate user intent of the plurality of candidate user intents.

13. The method of claim 8 , wherein:

a first candidate user intent of the plurality of candidate user intents is determined from a first candidate text representation of the plurality of candidate text representations; and

a second candidate user intent of the plurality of candidate user intents is determined from a second candidate text representation of the plurality of candidate text representations.

14. The method of claim 8 , further comprising:

ranking the plurality of candidate task flows in accordance with the plurality of task flow scores, wherein selecting the first candidate task flow is based on the ranking of the plurality of candidate task flows.

15. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:

receiving a user utterance;

determining, based on a plurality of candidate text representations of the user utterance, a plurality of candidate user intents for the user utterance, wherein each candidate user intent of the plurality of candidate user intents corresponds to a respective candidate task flow of a plurality of candidate task flows;

determining a plurality of task flow scores for the plurality of candidate task flows, each task flow score of the plurality of task flow scores corresponding to a respective candidate task flow of the plurality of candidate task flows;

selecting, based on the plurality of task flow scores, a first candidate task flow of the plurality of candidate task flows; and

executing the first candidate task flow, including presenting, to the user, results from executing the first candidate task flow.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the one or more programs further include instructions for:

for each candidate task flow of the plurality of candidate task flows:

resolving one or more flow parameters of the respective candidate task flow, wherein a respective task flow score for the respective candidate task flow is based on resolving the one or more flow parameters of the respective candidate task flow.

17. The non-transitory computer-readable storage medium of claim 16 , wherein resolving the one or more flow parameters of the respective candidate task flow comprises searching a data source for one or more values corresponding to the one or more flow parameters, the data source corresponding to one or more properties of a respective candidate user intent of the plurality of candidate user intents.

18. The non-transitory computer-readable storage medium of claim 15 , wherein:

each candidate text representation of the plurality of candidate text representations has an associated speech recognition confidence score; and

each task flow score of the plurality of task flow scores is based on a respective speech recognition confidence score of a respective candidate text representation of the plurality of candidate text representations.

19. The non-transitory computer-readable storage medium of claim 15 , wherein:

each candidate user intent of the plurality of candidate user intents has an associated intent confidence score; and

each task flow score of the plurality of task flow scores is based on a respective intent confidence score of a respective candidate user intent of the plurality of candidate user intents.

20. The non-transitory computer-readable storage medium of claim 15 , wherein:

a first candidate user intent of the plurality of candidate user intents is determined from a first candidate text representation of the plurality of candidate text representations; and

a second candidate user intent of the plurality of candidate user intents is determined from a second candidate text representation of the plurality of candidate text representations.

21. The non-transitory computer-readable storage medium of claim 15 , wherein the one or more programs further include instructions for:

ranking the plurality of candidate task flows in accordance with the plurality of task flow scores, wherein selecting the first candidate task flow is based on the ranking of the plurality of candidate task flows.

22. An electronic device, comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving a user utterance;

determining, based on a plurality of candidate text representations of the user utterance, a plurality of candidate user intents for the user utterance, wherein each candidate user intent of the plurality of candidate user intents corresponds to a respective candidate task flow of a plurality of candidate task flows, and wherein a first candidate task flow of the plurality of candidate task flows corresponds to a first candidate user intent of the plurality of candidate user intents;

determining a plurality of task flow scores for the plurality of candidate task flows, each task flow score of the plurality of task flow scores corresponding to a respective candidate task flow of the plurality of candidate task flows, wherein:

a first task flow score of the plurality of task flow scores is for the first candidate task flow;

determining the plurality of task flow scores includes determining a flow parameter value corresponding to a property of the first candidate user intent, the flow parameter value not specified in the user utterance; and

the first task flow score is based on whether the flow parameter value can be determined by the electronic device;

selecting, based on the plurality of task flow scores, the first candidate task flow; and

executing the first candidate task flow, including presenting, to the user, results from executing the first candidate task flow.

23. The electronic device of claim 22 , wherein the first task flow score is further based on a first intent confidence score of the first candidate user intent.

24. The electronic device of claim 23 , wherein the first task flow score is a highest task flow score of the plurality of task flow scores, and wherein the first intent confidence score is not a highest intent confidence score of a plurality of intent confidence scores that correspond to the plurality of candidate user intents.

25. The electronic device of claim 22 , wherein:

the first candidate user intent is determined from a first candidate text representation of the plurality of candidate text representations; and

the first task flow score is further based on a first speech recognition confidence score of the first candidate text representation.

26. The electronic device of claim 25 , wherein the first task flow score is a highest task flow score of the plurality of task flow scores, and wherein the first speech recognition confidence score is not a highest speech recognition confidence score of a plurality of speech recognition confidence scores that correspond to the plurality of candidate text representations.

Continuity (3)

Cited By (13)

US 12,197,817 US 12,200,297 US 12,236,952 US 12,301,635 US 12,333,404 US 12,361,943 US 12,367,879 US 12,386,434 US 12,386,491 US 12,477,470 US 12,562,172 US 12,608,171 US 12,626,702