Granted Patent B2

US 12,367,872 · App. 17/937,692 · Granted Jul 22, 2025

Inventors: Divya Neelagiri (Dublin, CA); Cindy Sushen Tseng (Santa Clara, CA); Vijendra Raj Apsingekar (San Jose, CA)

Assignee: Samsung Electronics Co., Ltd.

G10L15/197G10L15/005G10L15/22

Abstract

Claims (107)

1. A method performed by a first electronic device, the method comprising:

obtaining, from a second electronic device, an audio input of a person speaking and metadata available at the second electronic device, the audio input captured by the second electronic device, the metadata including at least one of:

a history of spoken language types associated with the person or the second electronic device; and

a keyboard language setting or a browser language setting associated with the second electronic device;

for each of multiple language types:

determining a first probability that the person is speaking in the language type by applying a trained spoken language identification model to the audio input;

determining at least one second probability that the person is speaking in the language type based on the metadata available at the second electronic device; and

determining a score for the language type based on a weighted sum of the first and second probabilities; and

identifying the language type associated with a highest score as a spoken language of the person in the audio input.

2. The method of claim 1 , wherein, for each of the language types, the at least one second probability comprises at least two of:

a probability of the person speaking in the language type based on a location associated with the person or the second electronic device;

a probability of the person speaking in the language type based on the history of spoken language types associated with the person or the second electronic device; and

a probability of the person speaking in the language type based on the keyboard language setting or the browser language setting associated with the second electronic device.

3. The method of claim 1 , wherein, for each of the language types, the at least one second probability comprises:

a probability of the person speaking in the language type based on a location associated with the person or the second electronic device;

a probability of the person speaking in the language type based on the history of spoken language types associated with the person or the second electronic device; and

a probability of the person speaking in the language type based on the keyboard language setting or the browser language setting associated with the second electronic device.

4. The method of claim 3 , wherein the history of spoken language types associated with the person includes a model trained using at least one of:

Mel frequency cepstral coefficients;

linear prediction cepstral coefficients representing vocal cord movements; and

wavelet octave coefficients of residues representing pitch, harmonic structure, and spectral energy distribution.

5. The method of claim 3 , wherein the metadata further includes the location associated with the person or the second electronic device.

6. The method of claim 1 , wherein the first electronic device provides a personalized spoken language identification service for multiple people who speak different language types using the trained spoken language identification model.

7. A method comprising:

obtaining an audio input of a person speaking, the audio input captured by an electronic device;

for each of multiple language types:

determining a first probability that the person is speaking in the language type by applying a trained spoken language identification model to the audio input;

determining at least one second probability that the person is speaking in the language type based on at least one characteristic of the person or the electronic device; and

determining a score for the language type based on a weighted sum of the first and second probabilities; and

identifying the language type associated with a highest score as a spoken language of the person in the audio input;

wherein, for each of the language types, the at least one second probability comprises:

a probability of the person speaking in the language type based on a location associated with the person or the electronic device;

a probability of the person speaking in the language type based on a history of spoken language types associated with the person or the electronic device; and

a probability of the person speaking in the language type based on a keyboard language setting or a browser language setting associated with the electronic device; and

wherein, for each of the language types, determining the score for the language type comprises using a formula of:

y *=argmax log P ( y|x )+ W loc P loc ( y )+ W speech P speech ( y )+ W text P text ( y ),

where:

y* represents the score;

P(y|x) represents the first probability;

P loc (y) represents the probability based on the location associated with the person;

P speech (y) represents the probability based on the history of spoken language types associated with the person;

P text (y) represents the probability based on the keyboard language setting or the browser language setting associated with the electronic device; and

W loc , W speech , and W text represent weights.

8. An apparatus comprising:

at least one processing device configured to:

obtain, from an electronic device, an audio input of a person speaking captured by the electronic device and metadata available at the electronic device, the metadata including at least one of:

a history of spoken language types associated with the person or the electronic device; and

a keyboard language setting or a browser language setting associated with the electronic device;

for each of multiple language types:

determine a first probability that the person is speaking in the language type by applying a trained spoken language identification model to the audio input;

determine at least one second probability that the person is speaking in the language type based on the metadata available at the electronic device; and

determine a score for the language type based on a weighted sum of the first and second probabilities; and

identify the language type associated with a highest score as a spoken language of the person in the audio input.

9. The apparatus of claim 8 , wherein, for each of the language types, the at least one second probability comprises at least two of:

a probability of the person speaking in the language type based on a location associated with the person or the electronic device;

a probability of the person speaking in the language type based on the history of spoken language types associated with the person or the electronic device; and

a probability of the person speaking in the language type based on the keyboard language setting or the browser language setting associated with the electronic device.

10. The apparatus of claim 8 , wherein, for each of the language types, the at least one second probability comprises:

a probability of the person speaking in the language type based on a location associated with the person or the electronic device;

a probability of the person speaking in the language type based on the history of spoken language types associated with the person or the electronic device; and

a probability of the person speaking in the language type based on the keyboard language setting or the browser language setting associated with the electronic device.

11. The apparatus of claim 10 , wherein, for each of the language types, the score for the language type is based on a formula of:

y *=argmax log P ( y|x )+ W loc P loc ( y )+ W speech P speech ( y )+ W text P text ( y ),

where:

y* represents the score;

P(y|x) represents the first probability;

P loc (y) represents the probability based on the location associated with the person;

P speech (y) represents the probability based on the history of spoken language types associated with the person;

P text (y) represents the probability based on the keyboard language setting or the browser language setting associated with the electronic device; and

W loc , W speech , and W text represent weights.

12. The apparatus of claim 10 , wherein the history of spoken language types associated with the person includes a model trained using at least one of:

Mel frequency cepstral coefficients;

linear prediction cepstral coefficients representing vocal cord movements; and

wavelet octave coefficients of residues representing pitch, harmonic structure, and spectral energy distribution.

13. The apparatus of claim 10 , wherein the metadata further includes the location associated with the person or the electronic device.

14. The apparatus of claim 8 , wherein the at least one processing device is configured to provide a personalized spoken language identification service for multiple people who speak different language types using the trained spoken language identification model.

15. A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:

obtain, from an electronic device, an audio input of a person speaking captured by the electronic device and metadata available at the electronic device, the metadata including at least one of:

a history of spoken language types associated with the person or the electronic device; and

a keyboard language setting or a browser language setting associated with the electronic device;

for each of multiple language types:

determine a first probability that the person is speaking in the language type by applying a trained spoken language identification model to the audio input;

determine at least one second probability that the person is speaking in the language type based on the metadata available at the electronic device; and

determine a score for the language type based on a weighted sum of the first and second probabilities; and

identify the language type associated with a highest score as a spoken language of the person in the audio input.

16. The non-transitory computer readable medium of claim 15 , wherein, for each of the language types, the at least one second probability comprises at least two of:

a probability of the person speaking in the language type based on a location associated with the person or the electronic device;

a probability of the person speaking in the language type based on the history of spoken language types associated with the person or the electronic device; and

a probability of the person speaking in the language type based on the keyboard language setting or the browser language setting associated with the electronic device.

17. The non-transitory computer readable medium of claim 15 , wherein, for each of the language types, the at least one second probability comprises:

a probability of the person speaking in the language type based on a location associated with the person or the electronic device;

a probability of the person speaking in the language type based on the history of spoken language types associated with the person or the electronic device; and

a probability of the person speaking in the language type based on the keyboard language setting or the browser language setting associated with the electronic device.

18. The non-transitory computer readable medium of claim 17 , wherein, for each of the language types, the score for the language type is based on a formula of:

y *=argmax log P ( y|x )+ W loc P loc ( y )+ W speech P speech ( y )+ W text P text ( y ),

where:

y* represents the score;

P(y|x) represents the first probability;

P loc (y) represents the probability based on the location associated with the person;

P speech (y) represents the probability based on the history of spoken language types associated with the person;

P text (y) represents the probability based on the keyboard language setting or the browser language setting associated with the electronic device; and

W loc , W speech , and W text represent weights.

19. The non-transitory computer readable medium of claim 17 , wherein the history of spoken language types associated with the person includes a model trained using at least one of:

Mel frequency cepstral coefficients;

linear prediction cepstral coefficients representing vocal cord movements; and

wavelet octave coefficients of residues representing pitch, harmonic structure, and spectral energy distribution.

20. The non-transitory computer readable medium of claim 15 , further containing instructions that when executed cause the at least one processor to provide a personalized spoken language identification service for multiple people who speak different language types using the trained spoken language identification model.

Assignments (1)

Continuity (2)

References Cited (55)