Granted Patent B2

US 12,288,036 · App. 17/464,560 · Granted Apr 29, 2025

Inventors: Benjamin T. H. Cox (Pittsburgh, PA); Antoine R. Raux (Los Gatos, CA)

Assignee: Apple Inc.

G06F40/35G06F9/4881G06F18/2113G06V30/274G10L15/1815

Abstract

Claims (172)

1. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:

receiving an utterance;

determining a natural language representation of the utterance;

determining a first software process associated with the natural language representation;

determining whether the natural language representation can be executed by a task flow of the first software process;

in accordance with a determination that the natural language representation cannot be executed by the task flow of the first software process:

determining a set of transformation instructions, wherein the set of transformation instructions include instructions to change the natural language representation to a revised natural language representation;

determining the revised natural language representation using the set of transformation instructions; and

providing the revised natural language representation to a second software process.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the natural language representation includes a user intent.

3. The non-transitory computer-readable storage medium of claim 1 , wherein the natural language representation includes an ontology.

4. The non-transitory computer-readable storage medium of claim 1 , wherein the first software process includes an application.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the first software process includes a plug-in associated with an application.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the first software process determines whether the natural language representation can be executed by a task flow of the first software process, the set of transformation instructions, and the revised natural language representation.

7. The non-transitory computer-readable storage medium of claim 1 , wherein, prior to providing the natural language representation to the first software process, determining whether the natural language representation can be executed by a task flow of the first software process, the set of transformation instructions, and the revised natural language representation.

8. The non-transitory computer-readable storage medium of claim 1 , wherein determining the set of transformation instructions further comprises:

determining a category of transformation instruction; and

retrieving the set of transformation instructions from the category of transformation instructions.

9. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further include instructions for:

in accordance with a determination that the natural language representation can be executed by the task flow of the first software process:

executing the task flow using the natural language representation; and

providing an output corresponding to the executed task flow.

10. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs further include instructions for:

storing the utterance and the natural language representation in association with the transformation instructions.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the utterance is a first utterance and wherein the one or more programs further include instructions for:

receiving a second utterance;

retrieving the natural language representation stored in association with the first utterance;

determining whether the natural language representation can be executed by the task flow of the first software process; and

in accordance with a determination that the natural language representation cannot be executed by the task flow of the first software process:

incrementing a count associated with the transformation instructions.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the one or more programs further include instructions for:

determining whether the count exceeds a predetermined threshold; and

in accordance with a determination that the count exceeds the predetermined threshold, training a digital assistant to automatically implement the transformation instructions.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the one or more programs further include instructions for:

after training the digital assistant to automatically implement the transformation instructions:

receiving a third utterance;

determining the revised natural language representation; and

providing the revised natural language representation to the second software process.

14. The non-transitory computer-readable storage medium of claim 12 , wherein the transformation instructions associated with the count that exceeds the predetermined threshold are determined to be salient.

15. The non-transitory computer-readable storage medium of claim 1 , wherein the transformation instructions include instructions to change a first user intent to a second user intent.

16. The non-transitory computer-readable storage medium of claim 1 , wherein the transformation instructions include instructions to map a user intent to an ontology.

17. The non-transitory computer-readable storage medium of claim 1 , wherein the task flow is a first task flow, the set of transformation instructions is a first set of transformation instructions, and the revised natural language representation is a first revised natural language representation and wherein the one or more programs further include instructions for:

determining whether the first revised natural language representation can be executed by a second task flow of the second software process;

in accordance with a determination that the first revised natural language representation cannot be executed by the second task flow:

determining a second set of transformation instructions;

determining a second revised natural language representation using the second set of transformation instructions; and

providing the second revised natural language representation to a third software process.

18. The non-transitory computer-readable storage medium of claim 1 , wherein determining the set of transformation instructions further comprises:

determining a plurality of transformation instructions associated with a plurality of software processes.

19. The non-transitory computer-readable storage medium of claim 18 , wherein determining a revised natural language representation using the set of transformation instructions further comprises:

determining a plurality of revised natural language representations corresponding to the plurality of transformer instructions;

ranking the plurality of revised natural language representations; and

selecting a highest ranked revised natural language representations as the revised natural language representation.

20. The non-transitory computer-readable storage medium of claim 19 , wherein ranking the plurality of revised natural language representations further comprises:

retrieving a plurality of natural language representations determined based on the utterance; and

ranking the plurality of natural language representations based on the transformation instructions.

21. An electronic device comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving an utterance;

determining a natural language representation of the utterance;

determining a first software process associated with the natural language representation;

determining whether the natural language representation can be executed by a task flow of the first software process;

in accordance with a determination that the natural language representation cannot be executed by the task flow of the first software process:

determining a set of transformation instructions, wherein the set of transformation instructions include instructions to change the natural language representation to a revised natural language representation;

determining the revised natural language representation using the set of transformation instructions; and

providing the revised natural language representation to a second software process.

22. A method, comprising:

at an electronic device with one or more processors and memory:

receiving an utterance;

determining a natural language representation of the utterance;

determining a first software process associated with the natural language representation;

determining whether the natural language representation can be executed by a task flow of the first software process;

in accordance with a determination that the natural language representation cannot be executed by the task flow of the first software process:

determining a set of transformation instructions, wherein the set of transformation instructions include instructions to change the natural language representation to a revised natural language representation;

determining the revised natural language representation using the set of transformation instructions; and

providing the revised natural language representation to a second software process.

23. The electronic device of claim 21 , wherein the natural language representation includes a user intent.

24. The electronic device of claim 21 , wherein the natural language representation includes an ontology.

25. The electronic device of claim 21 , wherein the first software process includes an application.

26. The electronic device of claim 21 , wherein the first software process includes a plug-in associated with an application.

27. The electronic device of claim 21 , wherein the first software process determines whether the natural language representation can be executed by a task flow of the first software process, the set of transformation instructions, and the revised natural language representation.

28. The electronic device of claim 21 , wherein, prior to providing the natural language representation to the first software process, determining whether the natural language representation can be executed by a task flow of the first software process, the set of transformation instructions, and the revised natural language representation.

29. The electronic device of claim 21 , wherein determining the set of transformation instructions further comprises:

determining a category of transformation instruction; and

retrieving the set of transformation instructions from the category of transformation instructions.

30. The electronic device of claim 21 , wherein the one or more programs further include instructions for:

in accordance with a determination that the natural language representation can be executed by the task flow of the first software process:

executing the task flow using the natural language representation; and

providing an output corresponding to the executed task flow.

31. The electronic device of claim 21 , wherein the one or more programs further include instructions for:

storing the utterance and the natural language representation in association with the transformation instructions.

32. The electronic device of claim 31 , wherein the utterance is a first utterance and wherein the one or more programs further include instructions for:

receiving a second utterance;

retrieving the natural language representation stored in association with the first utterance;

determining whether the natural language representation can be executed by the task flow of the first software process; and

in accordance with a determination that the natural language representation cannot be executed by the task flow of the first software process:

incrementing a count associated with the transformation instructions.

33. The electronic device of claim 32 , wherein the one or more programs further include instructions for:

determining whether the count exceeds a predetermined threshold; and

in accordance with a determination that the count exceeds the predetermined threshold, training a digital assistant to automatically implement the transformation instructions.

34. The electronic device of claim 33 , wherein the one or more programs further include instructions for:

after training the digital assistant to automatically implement the transformation instructions:

receiving a third utterance;

determining the revised natural language representation; and

providing the revised natural language representation to the second software process.

35. The electronic device of claim 33 , wherein the transformation instructions associated with the count that exceeds the predetermined threshold are determined to be salient.

36. The electronic device of claim 21 , wherein the transformation instructions include instructions to change a first user intent to a second user intent.

37. The electronic device of claim 21 , wherein the transformation instructions include instructions to map a user intent to an ontology.

38. The electronic device of claim 21 , wherein the task flow is a first task flow, the set of transformation instructions is a first set of transformation instructions, and the revised natural language representation is a first revised natural language representation and wherein the one or more programs further include instructions for:

determining whether the first revised natural language representation can be executed by a second task flow of the second software process;

in accordance with a determination that the first revised natural language representation cannot be executed by the second task flow:

determining a second set of transformation instructions;

determining a second revised natural language representation using the second set of transformation instructions; and

providing the second revised natural language representation to a third software process.

39. The electronic device of claim 21 , wherein determining the set of transformation instructions further comprises:

determining a plurality of transformation instructions associated with a plurality of software processes.

40. The electronic device of claim 39 , wherein determining a revised natural language representation using the set of transformation instructions further comprises:

determining a plurality of revised natural language representations corresponding to the plurality of transformer instructions;

ranking the plurality of revised natural language representations; and

selecting a highest ranked revised natural language representations as the revised natural language representation.

41. The electronic device of claim 40 , wherein ranking the plurality of revised natural language representations further comprises:

retrieving a plurality of natural language representations determined based on the utterance; and

ranking the plurality of natural language representations based on the transformation instructions.

42. The method of claim 22 , wherein the natural language representation includes a user intent.

43. The method of claim 22 , wherein the natural language representation includes an ontology.

44. The method of claim 22 , wherein the first software process includes an application.

45. The method of claim 22 , wherein the first software process includes a plug-in associated with an application.

46. The method of claim 22 , wherein the first software process determines whether the natural language representation can be executed by a task flow of the first software process, the set of transformation instructions, and the revised natural language representation.

47. The method of claim 22 , wherein, prior to providing the natural language representation to the first software process, determining whether the natural language representation can be executed by a task flow of the first software process, the set of transformation instructions, and the revised natural language representation.

48. The method of claim 22 , wherein determining the set of transformation instructions further comprises:

determining a category of transformation instruction; and

retrieving the set of transformation instructions from the category of transformation instructions.

49. The method of claim 22 , further comprising:

in accordance with a determination that the natural language representation can be executed by the task flow of the first software process:

executing the task flow using the natural language representation; and

providing an output corresponding to the executed task flow.

50. The method of claim 22 , further comprising:

storing the utterance and the natural language representation in association with the transformation instructions.

51. The method of claim 50 , wherein the utterance is a first utterance, the method further comprising:

receiving a second utterance;

retrieving the natural language representation stored in association with the first utterance;

determining whether the natural language representation can be executed by the task flow of the first software process; and

in accordance with a determination that the natural language representation cannot be executed by the task flow of the first software process:

incrementing a count associated with the transformation instructions.

52. The method of claim 51 , further comprising:

determining whether the count exceeds a predetermined threshold; and

in accordance with a determination that the count exceeds the predetermined threshold, training a digital assistant to automatically implement the transformation instructions.

53. The method of claim 52 , further comprising:

after training the digital assistant to automatically implement the transformation instructions:

receiving a third utterance;

determining the revised natural language representation; and

providing the revised natural language representation to the second software process.

54. The method of claim 52 , wherein the transformation instructions associated with the count that exceeds the predetermined threshold are determined to be salient.

55. The method of claim 22 , wherein the transformation instructions include instructions to change a first user intent to a second user intent.

56. The method of claim 22 , wherein the transformation instructions include instructions to map a user intent to an ontology.

57. The method of claim 22 , wherein the task flow is a first task flow, the set of transformation instructions is a first set of transformation instructions, and the revised natural language representation is a first revised natural language representation, the method further comprising:

determining whether the first revised natural language representation can be executed by a second task flow of the second software process;

in accordance with a determination that the first revised natural language representation cannot be executed by the second task flow:

determining a second set of transformation instructions;

determining a second revised natural language representation using the second set of transformation instructions; and

providing the second revised natural language representation to a third software process.

58. The method of claim 22 , wherein determining the set of transformation instructions further comprises:

determining a plurality of transformation instructions associated with a plurality of software processes.

59. The method of claim 58 , wherein determining a revised natural language representation using the set of transformation instructions further comprises:

determining a plurality of revised natural language representations corresponding to the plurality of transformer instructions;

ranking the plurality of revised natural language representations; and

selecting a highest ranked revised natural language representations as the revised natural language representation.

60. The method of claim 59 , wherein ranking the plurality of revised natural language representations further comprises:

retrieving a plurality of natural language representations determined based on the utterance; and

ranking the plurality of natural language representations based on the transformation instructions.

Assignments (1)

Continuity (2)

References Cited (47)