IP Library Granted Patent US 10,671,428
Granted Patent B2
US 10,671,428 · App. 15/166,090 · Granted Jun 2, 2020

Distributed personal assistant

Inventor: Nicolas Zeitlin (Berkeley, CA)
Assignee: Apple Inc.
G06F9/4843G06F3/167G06F9/453G06F9/5027G06F16/90332G10L15/285G10L15/30H04M3/4936G06F2209/509G06N3/006G10L15/22G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,671,428
App. No.
15/166,090
Filed
May 26, 2016
Granted
Jun 2, 2020
Kind
B2
Examiner
YEN, ERIC L
Art Unit
2658
USPC
704/275
Abstract

An exemplary method for using a virtual assistant may include, at an electronic device configured to transmit and receive data, receiving a user request for a service from a virtual assistant; determining at least one task to perform in response to the user request; estimating at least one performance characteristic for completion of the at least one task with the electronic device, based on at least one heuristic; based on the estimating, determining whether to execute the at least one task at the electronic device; in accordance with a determination to execute the at least one task at the electronic device, causing the execution of the at least one task at the electronic device; in accordance with a determination to execute the at least one task outside the electronic device: generating executable code for carrying out the least one task; and transmitting the executable code from the electronic device.

Claims (114)

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device, cause the electronic device to:

receive a user request for a service from a virtual assistant;

determine at least one task to perform in response to the user request;

estimate at least one performance characteristic for completion of the at least one task with the electronic device, based on at least one heuristic of a plurality of heuristics, wherein estimating the at least one performance characteristic based on the at least one heuristic of the plurality of heuristics comprises:

determining whether performing the at least one task requires using personal data stored on the electronic device;

in response to determining that performing the at least one task requires using personal data stored on the electronic device, identifying a first heuristic of the plurality of heuristics;

weighting a second heuristic of the plurality of heuristics relative to one or more other heuristics of the plurality of heuristics based on a first weight, wherein a value of the first weight is determined based on computing conditions associated with the electronic device; and

estimating the at least one performance characteristic based on the first heuristic and the weighted second heuristic;

based on the estimate, determine whether to execute the at least one task at the electronic device using the personal data;

in accordance with a determination to execute the at least one task at the electronic device, cause the execution of the at least one task at the electronic device; and

in accordance with a determination to execute the at least one task outside the electronic device:

generate executable code for carrying out the at least one task; and

transmit the executable code from the electronic device.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes the battery life of the electronic device.

3. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes availability at the electronic device of high-frequency wireless networking service compliant with the IEEE 802.11x standard.

4. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes network speed available to the electronic device.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes available memory space of the electronic device.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes processing speed of the electronic device.

7. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes the operating system of the electronic device.

8. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes available cache space at the electronic device.

9. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes the type of electronic device.

10. The non-transitory computer-readable storage medium of claim 1 , wherein the first heuristic includes privacy settings associated with the at least one task.

11. The non-transitory computer-readable storage medium of claim 1 , wherein the first heuristic includes data transmission restrictions associated with the at least one task.

12. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes a location of the electronic device.

13. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes the characteristics of the at least one task.

14. The non-transitory computer-readable storage medium of claim 1 , wherein the second heuristic includes user preferences associated with the task.

15. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one task includes context resolution.

16. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one task includes speech recognition.

17. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one task includes automatic speech recognition.

18. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one task includes domain classification.

19. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one task includes natural language parsing.

20. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one task includes query resolution.

21. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one task includes natural language synthesis.

22. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one task includes user interface resolution.

23. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one task includes text-to-speech conversion.

24. The non-transitory computer-readable storage medium of claim 1 , further comprising:

in response to the transmitting, receiving, at the electronic device, information associated with execution of the executable code.

25. The non-transitory computer-readable storage medium of claim 1 , further comprising, after the causing the execution, transmitting from the electronic device the results of the causing the execution.

26. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more programs comprise further instructions that cause the electronic device to:

estimate one or more performance characteristics for completion of at least two additional discrete tasks; and

determine, based on the estimating of the one or more performance characteristics for completion of the at least two additional discrete tasks, whether to execute the at least two additional discrete tasks at the electronic device or at a second electronic device.

27. The non-transitory computer-readable storage medium of claim 1 , wherein the causing the execution further comprises:

storing executable code on the electronic device;

wherein the causing the execution includes causing the execution of the stored code on the electronic device.

28. The non-transitory computer-readable storage medium of claim 1 , wherein the user request is in unstructured natural language format.

29. The non-transitory computer-readable storage medium of claim 1 , wherein the at least one performance characteristic includes an estimated duration to complete the at least one task.

30. The non-transitory computer-readable storage medium of claim 1 , wherein the virtual assistant is distributed between the electronic device and a server, and wherein transmitting the executable code comprises transmitting the executable code to the server.

31. The non-transitory computer-readable storage medium of claim 1 , wherein transmitting the executable code comprises transmitting the executable code to a second electronic device.

32. The non-transitory computer-readable storage medium of claim 31 , wherein the second electronic device is connected to the electronic device over a local network.

33. The non-transitory computer-readable storage medium of claim 1 , wherein transmitting the executable code comprises transmitting the executable code to a second electronic device;

further comprising fulfilling the user request at the second electronic device.

34. The non-transitory computer-readable storage medium of claim 1 , wherein the executable code enables a determination of whether information required for executing the at least one task is available to the virtual assistant.

35. A method of using a virtual assistant, comprising:

at an electronic device configured to transmit and receive data,

receiving a user request for a service from a virtual assistant;

determining at least one task to perform in response to the user request;

estimating at least one performance characteristic for completion of the at least one task with the electronic device, based on at least one heuristic of a plurality of heuristics, wherein estimating the at least one performance characteristic based on the at least one heuristic of the plurality of heuristics comprises:

determining whether performing the at least one task requires using personal data stored on the electronic device;

in response to determining that performing the at least one task requires using personal data stored on the electronic device, identifying a first heuristic of the plurality of heuristics;

weighting a second heuristic of the plurality of heuristics relative to one or more other heuristics of the plurality of heuristics based on a first weight, wherein a value of the first weight is determined based on computing conditions associated with the electronic device; and

estimating the at least one performance characteristic based on the first heuristic and the weighted second heuristic;

based on the estimating, determining whether to execute the at least one task at the electronic device;

in accordance with a determination to execute the at least one task at the electronic device, causing the execution of the at least one task at the electronic device using the personal data; and

in accordance with a determination to execute the at least one task outside the electronic device:

generating executable code for carrying out the at least one task; and

transmitting the executable code from the electronic device.

36. The method of claim 35 , further comprising:

in response to the transmitting, receiving, at the electronic device, information associated with execution of the executable code.

37. The method of claim 35 , further comprising, after the causing the execution, transmitting from the electronic device the results of the causing the execution.

38. The method of claim 35 , further comprising:

estimating one or more performance characteristics for completion of at least two additional discrete tasks; and

determining, based on the estimating of the one or more performance characteristics for completion of the at least two additional discrete tasks, whether to execute the at least two additional discrete tasks at the electronic device or at a second electronic device.

39. The method of claim 35 , wherein the causing the execution further comprises:

storing executable code on the electronic device;

wherein the causing the execution includes causing the execution of the stored code on the electronic device.

40. The method of claim 35 , wherein the user request is in unstructured natural language format.

41. The method of claim 35 , wherein the virtual assistant is distributed between the electronic device and a server, and wherein transmitting the executable code comprises transmitting the executable code to the server.

42. The method of claim 35 , wherein transmitting the executable code comprises transmitting the executable code to a second electronic device.

43. The method of claim 42 , wherein the second electronic device is connected to the electronic device over a local network.

44. The method of claim 35 , wherein transmitting the executable code comprises transmitting the executable code to a second electronic device;

further comprising fulfilling the user request at the second electronic device.

45. The method of claim 35 , wherein the executable code enables a determination of whether information required for executing the at least one task is available to the virtual assistant.

46. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving a user request for a service from a virtual assistant;

determining at least one task to perform in response to the user request;

estimating at least one performance characteristic for completion of the at least one task with the electronic device, based on at least one heuristic of a plurality of heuristics, wherein estimating the at least one performance characteristic based on the at least one heuristic of the plurality of heuristics comprises:

determining whether performing the at least one task requires using personal data stored on the electronic device;

in response to determining that performing the at least one task requires using personal data stored on the electronic device, identifying a first heuristic of the plurality of heuristics;

weighting a second heuristic of the plurality of heuristics relative to one or more other heuristics of the plurality of heuristics based on a first weight, wherein a value of the first weight is determined based on computing conditions associated with the electronic device; and

estimating the at least one performance characteristic based on the first heuristic and the weighted second heuristic;

based on the estimating, determining whether to execute the at least one task at the electronic device;

in accordance with a determination to execute the at least one task at the electronic device, causing the execution of the at least one task at the electronic device using the personal data; and

in accordance with a determination to execute the at least one task outside the electronic device:

generating executable code for carrying out the at least one task; and

transmitting the executable code from the electronic device.

47. The electronic device of claim 46 , wherein the one or more programs comprise further instructions for:

in response to the transmitting, receiving, at the electronic device, information associated with execution of the executable code.

48. The electronic device of claim 46 , wherein the one or more programs comprise further instructions for, after the causing the execution, transmitting from the electronic device the results of the causing the execution.

49. The electronic device of claim 46 , further comprising:

estimating one or more performance characteristics for completion of at least two additional discrete tasks; and

determining, based on the estimating of the one or more performance characteristics for completion of the at least two additional discrete tasks, whether to execute the at least two additional discrete tasks at the electronic device or at a second electronic device.

50. The electronic device of claim 46 , wherein the causing the execution further comprises:

storing executable code on the electronic device;

wherein the causing the execution includes causing the execution of the stored code on the electronic device.

51. The electronic device of claim 46 , wherein the user request is in unstructured natural language format.

52. The electronic device of claim 46 , wherein the virtual assistant is distributed between the electronic device and a server, and wherein transmitting the executable code comprises transmitting the executable code to the server.

53. The electronic device of claim 46 , wherein transmitting the executable code comprises transmitting the executable code to a second electronic device.

54. The electronic device of claim 53 , wherein the second electronic device is connected to the electronic device over a local network.

55. The electronic device of claim 46 , wherein transmitting the executable code comprises transmitting the executable code to a second electronic device;

further comprising fulfilling the user request at the second electronic device.

56. The electronic device of claim 46 , wherein the executable code enables a determination of whether information required for executing the at least one task is available to the virtual assistant.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 30, 2016
From: ZEITLIN, NICOLAS
To: APPLE INC.
Reel/Frame 039057/0558 →
Continuity (2)
Provisional Application 62215647 · Sep 8, 2015
Related Publication 20170068550A1 · Mar 9, 2017
Cited By (42)
US 12,197,712 US 12,197,817 US 12,197,857 US 12,200,297 US 12,204,932 US 12,211,502 US 12,216,894 US 12,219,314 US 12,223,282 US 12,236,952 US 12,254,887 US 12,260,234 US 12,266,380 US 12,277,954 US 12,293,203 US 12,293,763 US 12,301,635 US 12,327,559 US 12,333,404 US 12,361,943 US 12,367,879 US 12,379,894 US 12,380,281 US 12,380,876 US 12,386,434 US 12,386,491 US 12,399,877 US 12,431,128 US 12,477,470 US 12,556,890 US 12,567,415 US 12,579,374 US 12,608,171 US 12,613,621 US 12,613,730 US 12,619,452 US 12,620,179 US 12,633,289 US 12,640,142 US 12,640,151 US 12,670,639 US 12,675,839