Granted Patent B2

US 11,217,255 · App. 15/679,108 · Granted Jan 4, 2022

Inventors: Yoon Kim (Los Altos, CA); Charles Srisuwananukorn (Belmont, CA); David A. Carson (San Francisco, CA); Thomas R. Gruber (Emerald Hills, CA); Justin G. Binder (Oakland, CA)

Assignee: Apple Inc.

G10L17/22G10L15/16G10L15/22G10L15/30G10L15/187G10L15/26G10L2015/223

Abstract

Claims (97)

1. A method for providing a digital assistant service, comprising:

at a first electronic device with one or more processors and memory, the first electronic device being configured to extend speech-based services to a plurality of users based on identities of the plurality of users, wherein the speech-based services are provided by one or both of a second electronic device and a third electronic device:

receiving, from a first user, a first speech input representing a user request;

obtaining an identity of the first user after receiving the first speech input, wherein obtaining the identity of the first user is comprises:

obtaining authentication data, at the first electronic device, associated with the first user;

providing the authentication data, from the first electronic device, to at least one of a second electronic device or a third electronic device, which is a client device disposed in the vicinity of the first electronic device;

receiving the identity of the first user, at the first electronic device, from the third electronic device, wherein the identity of the first user is determined based on the authentication data and wherein receiving the identity of the first user from the third electronic device includes:

establishing a connection between the first electronic device and the third electronic device, wherein establishing the connection between the first electronic device and the third electronic device includes:

establishing the connection based on a near-field communication between the first electronic device and the third electronic device, or

establishing the connection based on detecting of the third electronic device being within a predetermined distance from the first electronic device;

in accordance with obtaining the identity of the first user, providing a representation of the user request to at least one of the second electronic device or the third electronic device;

receiving a response to the user request from the second electronic device or the third electronic device, wherein at least one of the second electronic device and the third electronic device determine to provide the response to the first electronic device;

determining, based on a second location determined from content of the first speech input and a first location of the first electronic device, that a representation of the response is to be provided by the second electronic device or the third electronic device located at the second location and not to be provided by the first electronic device; and

in response to determining that the representation of the response is to be provided by the second electronic device or the third electronic device and not to be provided by the first electronic device, forwarding, by the first electronic device, the response to the second electronic device or the third electronic device.

2. The method of claim 1 , wherein the second electronic device is a server remotely disposed from the first electronic device.

3. The method of claim 1 , wherein the representation of the user request to at least one of the second electronic device or the third electronic device is provided based on a type of the user request and a location of at least one of the second electronic device or the third electronic device, wherein the type of the user request is a user-specific information request or a non-user-specific information request.

4. The method of claim 1 , wherein providing the representation of the user request to at least one of the second electronic device or the third electronic device comprises:

determining whether the third electronic device is communicatively coupled to the first electronic device;

in accordance with a determination that the third electronic device is communicatively coupled to the first electronic device, providing the representation of the user request to the third electronic device and not to the second electronic device; and

in accordance with a determination that the third electronic device is not communicatively coupled to the first electronic device, providing the representation of the user request to the second electronic device.

5. The method of claim 1 , wherein providing the representation of the user request to at least one of the second electronic device or the third electronic device comprises:

providing the representation of the user request to the second electronic device and not to the third electronic device.

6. The method of claim 1 , wherein providing the representation of the user request to at least one of the second electronic device or the third electronic device comprises:

providing the representation of the user request to both the second electronic device and the third electronic device, wherein the representation of the user request includes a request for the second and the third electronic devices to determine whether the second and the third electronic devices are capable of providing the requested response; and

communicating, by the second electronic device and the third electronic device, capability determination to the first electronic device.

7. The method of claim 1 , wherein the representation of the user request is provided to the third electronic device and not to the second electronic device; and wherein receiving the response to the user request comprises:

causing the third electronic device to determine whether the third electronic device is capable of providing the response to the user request; and

in accordance with a determination that the third electronic device is capable of providing the response to the user request, receiving, at the first electronic device, the response to the user request from the third electronic device.

8. The method of claim 7 , further comprising:

in accordance with a determination that the third electronic device is incapable of providing the response to the user request,

forwarding, by the third electronic device, the representation of the user request to the second electronic device; and

receiving, at the first electronic device, the response to the user request from the second electronic device.

9. The method of claim 1 , wherein the representation of the user request is provided to the second electronic device and not to the third electronic device, and wherein receiving the response to the user request comprises:

causing the second electronic device to determine whether the second electronic device is capable of providing the response to the user request; and

in accordance with a determination that the second electronic device is capable of providing the response to the user request, receiving, at the first electronic device, the response to the user request from the second electronic device.

10. The method of claim 9 , further comprising:

in accordance with a determination that the second electronic device is incapable of providing the response to the user request,

forwarding, by the second electronic device, the representation of the user request to the third electronic device; and

receiving, at the first electronic device, the response to the user request.

11. The method of claim 10 , wherein the first electronic device receives the response to the user request from the third electronic device.

12. The method of claim 10 , wherein the first electronic device receives the response to the user request from the second electronic device based on a response provided by the third electronic device to the second electronic device.

13. The method of claim 1 , wherein the representation of the user request is provided from the first electronic device to both the second electronic device and the third electronic device, and wherein receiving the response to the user request comprises at least one of:

causing the second electronic device to determine whether the second electronic device is capable of providing the response to the user request; and

causing the third electronic device to determine whether the third electronic device is capable of providing the response to the user request.

14. The method of claim 13 , further comprising:

in accordance with a determination that the second electronic device is capable of providing the response to the user request, and that the third electronic device is incapable of providing the response to the user request,

receiving, at the first electronic device, the response to the user request from the second electronic device.

15. The method of claim 13 , further comprising:

in accordance with a determination that the third electronic device is capable of providing the response to the user request, and that the second electronic device is incapable of providing the response to the user request,

receiving, at the first electronic device, the response to the user request from the third electronic device.

16. The method of claim 13 , further comprising:

in accordance with a determination that both the second electronic device and the third electronic device are capable of providing the response to the user request,

receiving, at the first electronic device, the response to the user request from the second electronic device or the third electronic device based on a predetermined condition.

17. The method of claim 1 , wherein the second electronic device or the third electronic device provide a speech output including information in response to the user request.

18. The method of claim 1 , wherein the second electronic device or the third electronic device provide a speech output associated with performing a task in accordance with the user request.

19. The method of claim 18 , wherein the task is performed by:

the third electronic device;

the first electronic device and the third electronic device; or

the first electronic device, the third electronic device, and the second electronic device.

20. The method of claim 1 , wherein determining that the representation of the response is not to be provided by the first electronic device is further based on at least one of detecting a location of the user or tracking the user's movement.

21. The method of claim 1 , further comprising, prior to providing the representation of the user request to at least one of the second electronic device or the third electronic device:

establishing a connection between the first electronic device and at feast one of the second electronic device or the third electronic device.

22. The method of claim 21 , wherein establishing the connection is based on detecting of the third electronic device being within a predetermined distance from the first electronic device.

23. The method of claim 21 , further comprising:

establishing a connection between the first electronic device and the third electronic device, wherein the third electronic device is associated with the first user: and

establishing a connection between the first electronic device and a fourth electronic device, wherein the fourth electronic device is associated with a second user.

24. The method of claim 1 , further comprising, establishing one or more connections between the first electronic device and one or more electronic devices, wherein the one or more electronic devices are the same type of devices as the first electronic device.

25. The method of claim 1 , wherein at least one of the second electronic device and the third electronic device determine to provide the response to the first electronic device based on the respective physical capabilities of the second electronic device and the third electronic device.

26. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to:

receive, from a first user, a first speech input representing a user request by the first electronic device, the first electronic device being configured to extend speech-based services to a plurality of users based on identities of the plurality of users, wherein the speech-based services are provided by one or both of the second electronic device and the third electronic device;

obtaining an identity of the first user after receiving the first speech input, wherein obtaining the identity of the first user is comprises:

obtaining authentication data, at the first electronic device, associated with the first user;

providing the authentication data, from the first electronic device, to at least one of a second electronic device or a third electronic device, which is a client device disposed in the vicinity of the first electronic device;

receiving the identity of the first user, at the first electronic device, from the third electronic device, wherein the identity of the first user is determined based on the authentication data and wherein receiving the identity of the first user from the third electronic device includes:

establishing a connection between the first electronic device and the third electronic device, wherein establishing the connection between the first electronic device and the third electronic device includes:

establishing the connection based on a near-field communication between the first electronic device and the third electronic device, or

establishing the connection based on detecting of the third electronic device being within a predetermined distance from the first electronic device;

in accordance with obtaining the identity of the first user, provide a representation of the user request to at least one of the second electronic device or the third electronic device;

receive a response to the user request from the second electronic device or the third electronic device, wherein at least one of the second electronic device and the third electronic device determine to provide the response to the first electronic device;

determine, based on a second location determined from content of the first speech input and a first location of the first electronic device, that a representation of the response is to be provided by the second electronic device or the third electronic device located at the second location and not to be provided by the first electronic device; and

in response to determining that the representation of the response is to be provided by the second electronic device or the third electronic device and not to be provided by the first electronic device, forwarding, by the first electronic device, the response to the second electronic device or the third electronic device.

27. A first electronic device, comprising:

one or more processors;

memory; and

one or more programs stored in memory, the one or more programs including instructions for:

receiving, from a first user, a first speech input representing a user request by the first electronic device, the first electronic device being configured to extend speech-based services to a plurality of users based on identities of the plurality of users, wherein the speech-based services are provided by one or both of the second electronic device and the third electronic device;

obtaining an identity of the first user after receiving the first speech input, wherein obtaining the identity of the first user is comprises:

obtaining authentication data, at the first electronic device, associated with the first user;

providing the authentication data, from the first electronic device, to at least one of a second electronic device or a third electronic device, which is a client device disposed in the vicinity of the first electronic device;

receiving the identity of the first user, at the first electronic device, from the third electronic device, wherein the identity of the first user is determined based on the authentication data and wherein receiving the identity of the first user from the third electronic device includes:

establishing a connection between the first electronic device and the third electronic device, wherein establishing the connection between the first electronic device and the third electronic device includes:

establishing the connection based on a near-field communication between the first electronic device and the third electronic device, or

establishing the connection based on detecting of the third electronic device being within a predetermined distance from the first electronic device;

in accordance with obtaining the identity of the first user, providing a representation of the user request to at least one of the second electronic device or the third electronic device;

receiving a response to the user request from the second electronic device or the third electronic device, wherein at least one of the second electronic device and the third electronic device determine to provide the response to the first electronic device;

determining, based on a second location determined from content of the first speech input and a first location of the first electronic device, that a representation of the response is to be provided by the second electronic device or the third electronic device located at the second location and not to be provided by the first electronic device; and

in response to determining that the representation of the response is to be provided by the second electronic device or the third electronic device and not to be provided by the first electronic device, forwarding, by the first electronic device, the response to the second electronic device or the third electronic device.

Assignments (1)

Continuity (2)

Cited By (25)

US 12,197,817 US 12,200,297 US 12,211,502 US 12,216,894 US 12,219,314 US 12,236,952 US 12,249,338 US 12,254,887 US 12,260,234 US 12,277,954 US 12,293,763 US 12,301,635 US 12,333,404 US 12,361,943 US 12,367,879 US 12,380,876 US 12,386,434 US 12,386,491 US 12,477,470 US 12,525,241 US 12,585,966 US 12,608,171 US 12,619,452 US 12,640,151 US 12,651,020