Granted Patent B2

US 11,580,990 · App. 17/349,758 · Granted Feb 14, 2023

Inventors: Matthias Paulik (San Jose, CA); Henry G. Mason (San Francisco, CA); Jason A. Skinder (Bloomfield Hills, MI)

Assignee: Apple Inc.

G10L17/04G10L15/063G10L15/07G10L15/30G10L15/02G10L15/187G10L2015/0635G10L2015/0636

Abstract

Claims (105)

1. A method, comprising:

at an electronic device having one or more processors:

initiating a user-specific acoustic model on the electronic device;

receiving a plurality of speech inputs including a first speech input, each of the plurality of speech inputs associated with a user of the electronic device;

adjusting the user-specific acoustic model based on the plurality of speech inputs; and

providing the adjusted user-specific acoustic model to another electronic device; and

at the another electronic device:

receiving the adjusted user-specific acoustic model;

receiving a second speech input from a speaker;

identifying, with the adjusted user-specific acoustic model, the speaker of the second speech input as the user;

initiating a task based on the second speech input, wherein initiating the task includes executing a task flow; and

providing a result based on the initiated task, wherein the result includes an audio output.

2. The method of claim 1 , wherein providing the adjusted user-specific acoustic model to the another electronic device comprises:

determining whether the adjusted user-specific acoustic model has been trained on a threshold number of speech inputs;

in accordance with a determination that the user-specific acoustic model has been trained on the threshold number of speech inputs, providing the adjusted user-specific acoustic model to the another electronic device; and

in accordance with a determination that the adjusted user-specific acoustic model has not been trained on the threshold number of speech inputs:

further adjusting the adjusted user-specific model based on a second plurality of speech inputs and a plurality of speech results; and

providing the further adjusted user-specific acoustic model to the another electronic device.

3. The method of claim 1 , wherein identifying, with the adjusted user-specific acoustic model, the speaker of the second speech input comprises:

providing the second speech input to the adjusted user-specific acoustic model to provide a first speech result and a first accuracy score corresponding to the first speech result;

providing the second speech input to another user-specific acoustic model to provide a second speech result and a second accuracy score corresponding to the second speech result; and

identifying the speaker of the second speech input based on the first accuracy score and the second accuracy score.

4. The method of claim 1 , wherein receiving the plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs from the another electronic device.

5. The method of claim 1 , wherein receiving the plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs at the electronic device.

6. The method of claim 5 , wherein receiving the one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a phone call.

7. The method of claim 5 , wherein receiving the one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a request for a digital assistant.

8. The method of claim 1 , further comprising:

providing the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on a first predetermined portion of the plurality of speech inputs, wherein the user-independent acoustic model is based on a dataset, and wherein initializing the user-specific acoustic model comprises:

initializing the user-specific acoustic model using the dataset.

9. The method of claim 8 , wherein the user-independent acoustic model is an ensemble of two or more acoustic models.

10. The method of claim 1 , further comprising:

at the another electronic device, failing to identify, with the adjusted user-specific acoustic model, a second speaker of a third speech input as the user.

11. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

initiate a user-specific acoustic model on the electronic device;

receive a plurality of speech inputs including a first speech input, each of the plurality of speech inputs associated with a user of the electronic device;

adjust the user-specific acoustic model based on the plurality of speech inputs; and

provide the adjusted user-specific acoustic model to another electronic device; and

wherein the another electronic device:

receives the adjusted user-specific acoustic model,

receives a second speech input from a speaker;

identifies, with the adjusted user-specific acoustic model, the speaker of the second speech input as the user;

initiates a task based on the second speech input, wherein initiating the task includes executing a task flow; and

provides a result based on the initiated task, wherein the result includes an audio output.

12. The non-transitory computer-readable storage medium of claim 11 , wherein providing the adjusted user-specific acoustic model to the another electronic device comprises:

determining whether the adjusted user-specific acoustic model has been trained on a threshold number of speech inputs;

in accordance with a determination that the user-specific acoustic model has been trained on the threshold number of speech inputs, providing the adjusted user-specific acoustic model to the another electronic device; and

in accordance with a determination that the adjusted user-specific acoustic model has not been trained on the threshold number of speech inputs:

further adjusting the adjusted user-specific model based on a second plurality of speech inputs and a plurality of speech results; and

providing the further adjusted user-specific acoustic model to the another electronic device.

13. The non-transitory computer-readable storage medium of claim 11 , wherein identifying, with the adjusted user-specific acoustic model, the speaker of the second speech input comprises:

providing the second speech input to the adjusted user-specific acoustic model to provide a first speech result and a first accuracy score corresponding to the first speech result;

providing the second speech input to another user-specific acoustic model to provide a second speech result and a second accuracy score corresponding to the second speech result; and

identifying the speaker of the second speech input based on the first accuracy score and the second accuracy score.

14. The non-transitory computer-readable storage medium of claim 11 , wherein receiving the plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs from the another electronic device.

15. The non-transitory computer-readable storage medium of claim 11 , wherein receiving the plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs at the electronic device.

16. The non-transitory computer-readable storage medium of claim 15 , wherein receiving the one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a phone call.

17. The non-transitory computer-readable storage medium of claim 15 , wherein receiving the one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a request for a digital assistant.

18. The non-transitory computer-readable storage medium of claim 11 , wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:

provide the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on a first predetermined portion of the plurality of speech inputs, wherein the user-independent acoustic model is based on a dataset, and wherein initializing the user-specific acoustic model comprises:

initializing the user-specific acoustic model using the dataset.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the user-independent acoustic model is an ensemble of two or more acoustic models.

20. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

initiating a user-specific acoustic model on the electronic device;

receiving a plurality of speech inputs including a first speech input, each of the plurality of speech inputs associated with a user of the electronic device;

adjusting the user-specific acoustic model based on the plurality of speech inputs; and

providing the adjusted user-specific acoustic model to another electronic device; and

wherein the another electronic device:

receives the adjusted user-specific acoustic model;

receives a second speech input from a speaker;

identifies, with the adjusted user-specific acoustic model, the speaker of the second speech input as the user;

initiates a task based on the second speech input, wherein initiating the task includes executing a task flow; and

provides a result based on the initiated task, wherein the result includes an audio output.

21. The electronic device of claim 20 , wherein providing the adjusted user-specific acoustic model to the another electronic device comprises:

determining whether the adjusted user-specific acoustic model has been trained on a threshold number of speech inputs;

in accordance with a determination that the user-specific acoustic model has been trained on the threshold number of speech inputs, providing the adjusted user-specific acoustic model to the another electronic device; and

in accordance with a determination that the adjusted user-specific acoustic model has not been trained on the threshold number of speech inputs:

further adjusting the adjusted user-specific model based on a second plurality of speech inputs and a plurality of speech results; and

providing the further adjusted user-specific acoustic model to the another electronic device.

22. The electronic device of claim 20 , wherein identifying, with the adjusted user-specific acoustic model, the speaker of the second speech input comprises:

providing the second speech input to the adjusted user-specific acoustic model to provide a first speech result and a first accuracy score corresponding to the first speech result;

providing the second speech input to another user-specific acoustic model to provide a second speech result and a second accuracy score corresponding to the second speech result; and

identifying the speaker of the second speech input based on the first accuracy score and the second accuracy score.

23. The electronic device of claim 20 , wherein receiving the plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs from the another electronic device.

24. The electronic device of claim 20 , wherein receiving the plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs at the electronic device.

25. The electronic device of claim 24 , wherein receiving the one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a phone call.

26. The electronic device of claim 24 , wherein receiving the one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a request for a digital assistant.

27. The electronic device of claim 20 , wherein the one or more programs further include instructions for:

providing the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on a first predetermined portion of the plurality of speech inputs, wherein the user-independent acoustic model is based on a dataset, and wherein initializing the user-specific acoustic model comprises:

initializing the user-specific acoustic model using the dataset.

28. The electronic device of claim 27 , wherein the user-independent acoustic model is an ensemble of two or more acoustic models.

Continuity (4)

Cited By (15)

US 12,197,817 US 12,200,297 US 12,216,963 US 12,236,952 US 12,293,763 US 12,301,635 US 12,333,404 US 12,361,943 US 12,367,879 US 12,386,434 US 12,386,491 US 12,477,470 US 12,562,172 US 12,608,171 US 12,626,702