Granted Patent B2

US 10,410,637 · App. 15/713,276 · Granted Sep 10, 2019

Inventors: Matthias Paulik (San Jose, CA); Henry G. Mason (San Francisco, CA); Jason A. Skinder (Bloomfield Hills, MI)

Assignee: Apple Inc.

G10L17/04G10L15/063G10L15/07G10L15/30G10L15/02G10L15/187G10L2015/0635G10L2015/0636

Abstract

Claims (101)

1. A plurality of non-transitory computer-readable storage media storing one or more programs, wherein the one or more programs comprise instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to:

receive a plurality of speech inputs, each of the speech inputs associated with a same user of the first electronic device;

provide each of the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on the plurality of speech inputs;

initiate a user-specific acoustic model on the first electronic device;

adjust the user-specific acoustic model based on the plurality of speech inputs and the plurality of speech results;

provide the adjusted user-specific acoustic model to a second electronic device; wherein the one or more programs comprise instructions, which when executed by one or more processors of the second electronic device, cause the second electronic device to:

receive the adjusted user-specific acoustic model;

receive a second speech input; and

identify, with the adjusted user-specific acoustic model, a speaker of the second speech input.

2. The plurality of non-transitory computer-readable storage media of claim 1 , wherein providing the adjusted user-specific acoustic model to the second electronic device comprises:

determining whether the adjusted user-specific acoustic model has been trained on a threshold number of speech inputs;

in accordance with a determination that the adjusted user-specific acoustic model has been trained on a threshold number of speech inputs, providing the adjusted user-specific acoustic model to the second electronic device; and

in accordance with a determination that the adjusted user-specific acoustic model has not been trained on a threshold number of speech inputs:

further adjusting the adjusted user-specific model based on a second plurality of speech inputs and a second plurality of speech results; and

providing the further adjusted user-specific acoustic model to the second electronic device.

3. The plurality of non-transitory computer-readable storage media of claim 1 , wherein identifying, with the adjusted user-specific acoustic model, a speaker of the second speech input comprises:

providing the second speech input to the adjusted user-specific acoustic model to provide a first speech result and a first accuracy score corresponding to the first speech result;

providing the second speech input to another user-specific acoustic model to provide a second speech result and a second accuracy score corresponding to the second speech result; and

identifying the speaker of the second speech input based on the first accuracy score and the second accuracy score.

4. The plurality of non-transitory computer-readable storage media of claim 1 , wherein receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the first electronic device comprises:

receiving one or more speech inputs of the plurality of speech inputs from the second electronic device.

5. The plurality of non-transitory computer-readable storage media of claim 1 , wherein receiving a plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs at the first electronic device.

6. The plurality of non-transitory computer-readable storage media of claim 5 , wherein receiving one or more speech inputs of the plurality of speech inputs at the first electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a phone call.

7. The plurality of non-transitory computer-readable storage media of claim 5 , wherein receiving one or more speech inputs of the plurality of speech inputs at the first electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a request for a digital assistant.

8. The plurality of non-transitory computer-readable storage media of claim 1 , wherein the user-independent acoustic model is based on a dataset, and

wherein the user-specific acoustic model

is initiated using the dataset.

9. The plurality of non-transitory computer-readable storage media of claim 1 , wherein the user-independent acoustic model has a first number of parameters and the adjusted user-specific acoustic model has a second number of parameters, wherein the first number is greater than the second number.

10. The plurality of non-transitory computer-readable storage media of claim 1 , wherein the user-independent acoustic model is an ensemble of two or more acoustic models.

11. A method, comprising:

at a first electronic device having one or more processors:

receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the first electronic device;

providing each of the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on the plurality of speech inputs;

initiating a user-specific acoustic model on the first electronic device;

adjusting the user-specific acoustic model based on the plurality of speech inputs and the plurality of speech results;

providing the adjusted user-specific acoustic model to a second electronic device; wherein at the second electronic device:

receiving the adjusted user-specific acoustic model;

receiving a second speech input; and

identifying, with the adjusted user-specific acoustic model, a speaker of the second speech input.

12. The method of claim 11 , wherein providing the adjusted user-specific acoustic model to the second electronic device comprises:

determining whether the adjusted user-specific acoustic model has been trained on a threshold number of speech inputs;

in accordance with a determination that the adjusted user-specific acoustic model has been trained on a threshold number of speech inputs, providing the adjusted user-specific acoustic model to the second electronic device; and

in accordance with a determination that the adjusted user-specific acoustic model has not been trained on a threshold number of speech inputs:

further adjusting the adjusted user-specific model based on a second plurality of speech inputs and a second plurality of speech results; and

providing the further adjusted user-specific acoustic model to the second electronic device.

13. The method of claim 11 , wherein identifying, with the adjusted user-specific acoustic model, a speaker of the second speech input comprises:

providing the second speech input to the adjusted user-specific acoustic model to provide a first speech result and a first accuracy score corresponding to the first speech result;

providing the second speech input to another user-specific acoustic model to provide a second speech result and a second accuracy score corresponding to the second speech result; and

identifying the speaker of the second speech input based on the first accuracy score and the second accuracy score.

14. The method of claim 11 , wherein receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the first electronic device comprises:

receiving one or more speech inputs of the plurality of speech inputs from the second electronic device.

15. The method of claim 11 , wherein receiving a plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs at the first electronic device.

16. The method of claim 15 , wherein receiving one or more speech inputs of the plurality of speech inputs at the first electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a phone call.

17. The method of claim 15 , wherein receiving one or more speech inputs of the plurality of speech inputs at the first electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a request for a digital assistant.

18. The method of claim 13 , wherein the user-independent acoustic model is based on a dataset, and

wherein the user-specific acoustic model

is initiated using the dataset.

19. The method of claim 11 , wherein the user-independent acoustic model has a first number of parameters and the adjusted user-specific acoustic model has a second number of parameters, wherein the first number is greater than the second number.

20. The method of claim 11 , wherein the user-independent acoustic model is an ensemble of two or more acoustic models.

21. A system, comprising:

a first electronic device, comprising:

one or more first processors; a first memory; and one or more first programs, wherein the one or more first programs are stored in the first memory and configured to be executed by the one or more first processors, the one or more first programs including instructions for:

receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the first electronic device;

providing each of the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on the plurality of speech inputs;

initiating a user-specific acoustic model on the first electronic device;

adjusting the user-specific acoustic model based on the plurality of speech inputs and the plurality of speech results;

providing the adjusted user-specific acoustic model to a second electronic device; and

the second electronic device, comprising:

one or more second processors; a second memory; and one or more second programs, wherein the one or more second programs are stored in the second memory and configured to be executed by the one or more second processors, the one or more second programs including instructions for:

receiving the adjusted user-specific acoustic model;

receiving a second speech input; and

identifying, with the adjusted user-specific acoustic model, a speaker of the second speech input.

22. The system of claim 21 , wherein providing the adjusted user-specific acoustic model to the second electronic device comprises:

determining whether the adjusted user-specific acoustic model has been trained on a threshold number of speech inputs;

in accordance with a determination that the adjusted user-specific acoustic model has been trained on a threshold number of speech inputs, providing the adjusted user-specific acoustic model to the second electronic device; and

in accordance with a determination that the adjusted user-specific acoustic model has not been trained on a threshold number of speech inputs:

further adjusting the adjusted user-specific model based on a second plurality of speech inputs and a second plurality of speech results; and

providing the further adjusted user-specific acoustic model to the second electronic device.

23. The system of claim 21 , wherein identifying, with the adjusted user-specific acoustic model, a speaker of the second speech input comprises:

providing the second speech input to the adjusted user-specific acoustic model to provide a first speech result and a first accuracy score corresponding to the first speech result;

providing the second speech input to another user-specific acoustic model to provide a second speech result and a second accuracy score corresponding to the second speech result; and

identifying the speaker of the second speech input based on the first accuracy score and the second accuracy score.

24. The system of claim 21 , wherein receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the first electronic device comprises:

receiving one or more speech inputs of the plurality of speech inputs from the second electronic device.

25. The system of claim 21 , wherein receiving a plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs at the first electronic device.

26. The system of claim 25 , wherein receiving one or more speech inputs of the plurality of speech inputs at the first electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a phone call.

27. The system of claim 25 , wherein receiving one or more speech inputs of the plurality of speech inputs at the first electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a request for a digital assistant.

28. The system of claim 21 , wherein the user-independent acoustic model is based on a dataset, and

wherein the user-specific acoustic model

is initiated using the dataset.

29. The system of claim 21 , wherein the user-independent acoustic model has a first number of parameters and the adjusted user-specific acoustic model has a second number of parameters, wherein the first number is greater than the second number.

30. The system of claim 21 , the user-independent acoustic model is an ensemble of two or more acoustic models.

Assignments (2)

Continuity (2)

Cited By (30)

US 12,190,873 US 12,197,712 US 12,197,817 US 12,200,297 US 12,204,932 US 12,211,502 US 12,216,894 US 12,219,314 US 12,223,282 US 12,236,952 US 12,249,320 US 12,260,234 US 12,277,954 US 12,293,203 US 12,293,763 US 12,293,764 US 12,301,635 US 12,333,404 US 12,361,943 US 12,367,879 US 12,380,876 US 12,386,434 US 12,386,491 US 12,431,128 US 12,477,470 US 12,556,890 US 12,608,171 US 12,613,730 US 12,619,452 US 12,651,020