Digital assistant services based on device capabilities
Systems and processes for operating an intelligent automated assistant to provide extension of digital assistant services are provided. An example method includes, at an electronic device having one or more processors, receiving, from a first user, a first speech input representing a user request. The method further includes obtaining an identity of the first user; and in accordance with the user identity, providing a representation of the user request to at least one of a second electronic device or a third electronic device. The method further includes receiving, based on a determination of whether the second electronic device or the third electronic device, or both, is to provide the response to the first electronic device, the response to the user request from the second electronic device or the third electronic device. The method further includes providing a representation of the response to the first user.
1. A method for providing a digital assistant service, comprising:
at a first electronic device with one or more processors and memory:
receiving, from a first user, a first speech input representing a user request;
obtaining capability data associated with a plurality of electronic devices capable of being communicatively coupled to the first electronic device;
in accordance with obtaining the capability data:
obtaining, based on the first speech input, a plurality of steps for responding to the user request; and
identifying, from the plurality of electronic devices capable of being communicatively coupled to the first electronic device:
a second electronic device for performing a portion of the plurality of steps for responding to the user request, wherein performing a first step of the portion of the plurality of steps requires performing a second step that alters a state of the second electronic device prior to performing the first step; and
one or more additional electronic devices for performing the remaining steps of the plurality of steps for responding to the user request;
determining whether performing each step of the plurality of steps would alter a state of a respective electronic device, including determining that performing the second step would alter the state of the second electronic device;
in response to determining that performing the second step would alter the state of the second electronic device:
providing a first speech output to the first user regarding performing the first step by the second electronic device;
receiving a second speech input from the first user;
in response to receiving the second speech input, determining that the second electronic device is to perform the first step; and
causing the second electronic device and the one or more additional electronic devices to perform the respective steps in response to the user request, wherein the respective steps performed by the second electronic device and the one or more additional electronic devices are different steps.
2. The method of claim 1 , wherein obtaining the capability data comprises:
obtaining device capabilities associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
3. The method of claim 2 , wherein the device capabilities comprise one or more physical attributes associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
4. The method of claim 2 , wherein the device capabilities comprise data that are capable of being provided by the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
5. The method of claim 1 , wherein the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device are disposed within a predetermined boundary.
6. The method of claim 1 , wherein obtaining the plurality of steps for responding to the user request comprises at least one of:
receiving, from a third electronic device remotely located from the first electronic device, a plan for responding to the user request; and
determining, by the first electronic device, a second plan for responding to the user request, wherein the second plan comprises the plurality of steps for responding to the user request.
7. The method of claim 1 , wherein determining whether performing each step of the plurality of steps would alter a state of a respective electronic device comprises:
annotating one or more steps for responding to the user request, and
determining that performing the second step would alter the state of the second electronic device based on the annotation.
8. The method of claim 1 , further comprising, prior to obtaining capability data associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device:
establishing a connection between the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
9. The method of claim 8 , wherein establishing the connection is based on a near-field communication between the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
10. The method of claim 8 , wherein establishing the connection is based on detecting of the plurality of electronic devices capable of being communicatively coupled to the first electronic device being within a predetermined distance from the first electronic device.
11. The method of claim 8 , wherein establishing the connection is based on one or more previous established connections between the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
12. The method of claim 1 , further comprising, prior to obtaining capability data associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device:
inquiring the first user regarding accessing, by the first electronic device, the plurality of electronic devices capable of being communicatively coupled to the first electronic device;
receiving a third speech input from the first user; and
in response to receiving the third speech input, determining whether the first electronic device is authorized to access the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
13. The method of claim 12 , further comprising:
providing one or more duration options for accessing the plurality of electronic devices capable of being communicatively coupled to the first electronic device;
receiving a selection of a duration option from the first user; and
accessing the plurality of electronic devices capable of being communicatively coupled to the first electronic device based on the selected duration option.
14. The method of claim 1 , further comprising, prior to causing the second electronic device and the one or more additional electronic devices to perform the respective steps in response to the user request:
obtaining an identity of the first user; and
determining, based on the identity of the first user, whether the first user is authorized to receive at least a second portion of the response to the user request.
15. The method of claim 14 , wherein obtaining the identity of the first user comprises obtaining the identity based on a voice profile.
16. A non-transitory computer-readable storage medium comprising one or more programs for execution by one or more processors of a first electronic device, the one or more programs including instructions which, when executed by the one or more processors, cause the first electronic device to perform:
receiving, from a first user, a first speech input representing a user request,
obtaining capability data associated with a plurality of electronic devices capable of being communicatively coupled to the first electronic device;
in accordance with obtaining the capability data:
obtaining, based on the first speech input, a plurality of steps for responding to the user request; and
identifying, from the plurality of electronic devices capable of being communicatively coupled to the first electronic device:
a second electronic device for performing a portion of the plurality of steps for responding to the user request, wherein performing a first step of the portion of the plurality of steps requires performing a second step that alters a state of the second electronic device prior to performing the first step; and
one or more additional electronic devices for performing the remaining steps of the plurality of steps for responding to the user request;
determining whether performing each step of the plurality of steps would alter a state of a respective electronic device, including determining that performing the second step would alter the state of the second electronic device;
in response to determining that performing the second step would alter the state of the second electronic device:
providing a first speech output to the first user regarding performing the first step by the second electronic device;
receiving a second speech input from the first user;
in response to receiving the second speech input, determining that the second electronic device is to perform the first step; and
causing the second electronic device and the one or more additional electronic devices to perform the respective steps in response to the user request, wherein the respective steps performed by the second electronic device and the one or more additional electronic devices are different steps.
17. The non-transitory computer-readable storage medium of claim 16 , wherein obtaining the capability data comprises:
obtaining device capabilities associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the device capabilities comprise one or more physical attributes associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
19. The non-transitory computer-readable storage medium of claim 17 , wherein the device capabilities comprise data that are capable of being provided by the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
20. The non-transitory computer-readable storage medium of claim 16 , wherein the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device are disposed within a predetermined boundary.
21. The non-transitory computer-readable storage medium of claim 16 , wherein obtaining the plurality of steps for responding to the user request comprises at least one of:
receiving, from a third electronic device remotely located from the first electronic device, a plan for responding to the user request; and
determining, by the first electronic device, a second plan for responding to the user request, wherein the second plan comprises the plurality of steps for responding to the user request.
22. The non-transitory computer-readable storage medium of claim 16 , wherein determining whether performing each step of the plurality of steps would alter a state of a respective electronic device comprises:
annotating one or more steps for responding to the user request; and
determining that performing the second step would alter the state of the second electronic device based on the annotation.
23. The non-transitory computer-readable storage medium of claim 16 , wherein the one or more programs further include instructions which, when executed by the one or more processors, cause the first electronic device to perform:
prior to obtaining capability data associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device:
establishing a connection between the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
24. The non-transitory computer-readable storage medium of claim 23 , wherein establishing the connection is based on a near-field communication between the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
25. The non-transitory computer-readable storage medium of claim 23 , wherein establishing the connection is based on detecting of the plurality of electronic devices capable of being communicatively coupled to the first electronic device being within a predetermined distance from the first electronic device.
26. The non-transitory computer-readable storage medium of claim 23 , wherein establishing the connection is based on one or more previous established connections between the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
27. The non-transitory computer-readable storage medium of claim 16 , wherein the one or more programs further include instructions which, when executed by the one or more processors, cause the first electronic device to perform:
prior to obtaining capability data associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device:
inquiring the first user regarding accessing, by the first electronic device, the plurality of electronic devices capable of being communicatively coupled to the first electronic device;
receiving a third speech input from the first user; and
in response to receiving the third speech input, determining whether the first electronic device is authorized to access the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
28. The non-transitory computer-readable storage medium of claim 27 , wherein the one or more programs further include instructions which, when executed by the one or more processors, cause the first electronic device to perform:
providing one or more duration options for accessing the plurality of electronic devices capable of being communicatively coupled to the first electronic device;
receiving a selection of a duration option from the first user; and
accessing the plurality of electronic devices capable of being communicatively coupled to the first electronic device based on the selected duration option.
29. The non-transitory computer-readable storage medium of claim 16 , wherein the one or more programs further include instructions which, when executed by the one or more processors, cause the first electronic device to perform:
prior to causing the second electronic device and the one or more additional electronic devices to perform the respective steps in response to the user request:
obtaining an identity of the first user; and
determining, based on the identity of the first user, whether the first user is authorized to receive at least a second portion of the response to the user request.
30. The non-transitory computer-readable storage medium of claim 29 , wherein obtaining the identity of the first user comprises obtaining the identity based on a voice profile.
31. A first electronic device, comprising:
one or more processors;
memory; and
one or more programs stored in memory, the one or more programs including instructions for:
receiving, from a first user, a first speech input representing a user request;
obtaining capability data associated with a plurality of electronic devices capable of being communicatively coupled to the first electronic device;
in accordance with obtaining the capability data:
obtaining, based on the first speech input, a plurality of steps for responding to the user request; and
identifying, from the plurality of electronic devices capable of being communicatively coupled to the first electronic device:
a second electronic device for performing a portion of the plurality of steps for responding to the user request, wherein performing a first step of the portion of the plurality of steps requires performing a second step that alters a state of the second electronic device prior to performing the first step; and
one or more additional electronic devices for performing the remaining steps of the plurality of steps for responding to the user request;
determining whether performing each step of the plurality of steps would alter a state of a respective electronic device, including determining that performing the second step would alter the state of the second electronic device;
in response to determining that performing the second step would alter the state of the second electronic device:
providing a first speech output to the first user regarding performing the first step by the second electronic device;
receiving a second speech input from the first user; and
in response to receiving the second speech input, determining that the second electronic device is to perform the first step;
causing the second electronic device and the one or more additional electronic devices to perform the respective steps in response to the user request, wherein the respective steps performed by the second electronic device and the one or more additional electronic devices are different steps.
32. The first electronic device of claim 31 , wherein obtaining the capability data comprises:
obtaining device capabilities associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
33. The first electronic device of claim 32 , wherein the device capabilities comprise one or more physical attributes associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
34. The first electronic device of claim 32 , wherein the device capabilities comprise data that are capable of being provided by the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
35. The first electronic device of claim 31 , wherein the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device are disposed within a predetermined boundary.
36. The first electronic device of claim 31 , wherein obtaining the plurality of steps for responding to the user request comprises at least one of:
receiving, from a third electronic device remotely located from the first electronic device, a plan for responding to the user request; and
determining, by the first electronic device, a second plan for responding to the user request, wherein the second plan comprises the plurality of steps for responding to the user request.
37. The first electronic device of claim 31 , wherein determining whether performing each step of the plurality of steps would alter a state of a respective electronic device comprises:
annotating one or more steps for responding to the user request; and
determining that performing the second step would alter the state of the second electronic device based on the annotation.
38. The first electronic device of claim 31 , wherein the one or more programs further include instructions for:
prior to obtaining capability data associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device:
establishing a connection between the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
39. The first electronic device of claim 38 , wherein establishing the connection is based on a near-field communication between the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
40. The first electronic device of claim 38 , wherein establishing the connection is based on detecting of the plurality of electronic devices capable of being communicatively coupled to the first electronic device being within a predetermined distance from the first electronic device.
41. The first electronic device of claim 38 , wherein establishing the connection is based on one or more previous established connections between the first electronic device and the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
42. The first electronic device of claim 31 , wherein the one or more programs further include instructions for:
prior to obtaining capability data associated with the plurality of electronic devices capable of being communicatively coupled to the first electronic device:
inquiring the first user regarding accessing, by the first electronic device, the plurality of electronic devices capable of being communicatively coupled to the first electronic device;
receiving a third speech input from the first user; and
in response to receiving the third speech input, determining whether the first electronic device is authorized to access the plurality of electronic devices capable of being communicatively coupled to the first electronic device.
43. The first electronic device of claim 42 , wherein the one or more programs further include instructions for:
providing one or more duration options for accessing the plurality of electronic devices capable of being communicatively coupled to the first electronic device;
receiving a selection of a duration option from the first user; and
accessing the plurality of electronic devices capable of being communicatively coupled to the first electronic device based on the selected duration option.
44. The first electronic device of claim 31 , wherein the one or more programs further include instructions for:
prior to causing the second electronic device and the one or more additional electronic devices to perform the respective steps in response to the user request:
obtaining an identity of the first user; and
determining, based on the identity of the first user, whether the first user is authorized to receive at least a second portion of the response to the user request.
45. The first electronic device of claim 44 , wherein obtaining the identity of the first user comprises obtaining the identity based on a voice profile.