Granted Patent B2

US 12,361,934 · App. 17/864,937 · Granted Jul 15, 2025

Inventors: Jacob Assa (New York, NY); Alan Bekker (Givaat Shmuel, IL); Zach Moshe (Tel Aviv, IL)

Assignee: Snap Inc.

G10L15/197G10L15/10G10L15/187G10L15/22

Abstract

Claims (73)

1. A method comprising:

accessing, by one or more processors of an automatic speech recognition (ASR) engine, a language model (LM) that includes a plurality of n-grams, each of the plurality of n-grams comprising a respective sequence of words and corresponding LM score, the ASR engine comprising an acoustic component and a decoder;

converting, by the acoustic component, voice input into logits of inferred characters of phonemes;

selecting a target word to boost in the LM;

receiving input comprising a boosting factor including a specific boosting value for the target word;

identifying a target n-gram in the LM that includes the target word;

identifying a subset of n-grams of the plurality of n-grams that includes words in a portion of the target n-gram excluding the target word;

within the subset of n-grams that exclude the target word, identifying an individual n-gram having an individual LM score that corresponds to the specific boosting value of the boosting factor received in the input;

adjusting the LM score of the target n-gram using the individual LM score of the individual n-gram, within the subset of n-grams that exclude the target word, that corresponds to the specific boosting value of the boosting factor received in the input; and

processing, by the decoder, the logits of inferred characters of phonemes using the LM comprising the adjusted LM score of the target n-gram to generate a transcription of the voice input.

2. The method of claim 1 , wherein the target word is a suffix of the target n-gram, wherein portions of the target n-gram leading up to the suffix comprise a prefix of the target n-gram, wherein identifying the subset of n-grams comprises searching the plurality of n-grams to identify n-grams comprising the prefix of the target n-gram.

3. The method of claim 1 , wherein the boosting factor comprises a percentile.

4. The method of claim 3 , further comprising:

obtaining the LM score of the target n-gram; and

generating a group of n-grams of the subset of n-grams that are each associated with a LM score that is greater than the LM score of the target n-gram.

5. The method of claim 4 , further comprising:

computing a distribution of LM scores of the group of n-grams;

selecting a target LM score within the distribution of LM scores that corresponds to the percentile of the boosting factor; and

replacing the LM score of the target n-gram with the target LM score.

6. The method of claim 3 , further comprising:

computing a distribution of LM scores of the subset of n-grams;

selecting a target LM score within the distribution of LM scores that corresponds to the percentile of the boosting factor; and

replacing the LM score of the target n-gram with the target LM score.

7. The method of claim 1 , further comprising:

determining that the target n-gram is associated with a token corresponding to a start of a sentence, wherein the LM score of the target n-gram is adjusted based on determining that the target n-gram is associated with the token corresponding to the start of the sentence.

8. The method of claim 7 , further comprising:

preventing the LM score of the target n-gram from being adjusted in response to determining that the target n-gram is not associated with the token corresponding to the start of the sentence.

9. The method of claim 1 , further comprising:

receiving a given word that is excluded from the language model;

computing similarity values between the given word and each of the plurality of words in the language model; and

selecting the target word based on determining that the similarity value of the target word transgresses a similarity threshold.

10. The method of claim 9 , wherein the target word and the given word are synonyms.

11. The method of claim 9 , further comprising:

replacing the target word in the target n-gram with the given word to generate a new n-gram; and

associating the new n-gram with a LM score computed based on the LM scores of the subset of n-grams and the boosting factor.

12. The method of claim 1 , further comprising:

receiving a given word that is excluded from the language model;

identifying a collection of unigrams in the language model;

computing a distribution of LM scores of the collection of unigrams; and

selecting a given LM score from the distribution of LM scores corresponding to the boosting factor.

13. The method of claim 12 , further comprising:

generating a new n-gram comprising the given word; and

associating the given LM score with the new n-gram.

14. The method of claim 1 , further comprising:

obtaining acoustic details of each of the plurality of n-grams;

generating acoustic details of the target word; and

computing acoustic similarity values between the acoustic details of the target word and the acoustic details of each of the plurality of n-grams.

15. The method of claim 14 , further comprising:

identifying the target n-gram from the plurality of n-grams that has acoustic similarity values that transgress an acoustic similarity threshold.

16. The method of claim 14 , wherein the acoustic similarity values are computed based on phoneme distances between phonemes of the plurality of n-grams and phonemes of the target word.

17. The method of claim 1 , wherein the target word is selected in real-time for temporary use in response to receiving a user request to access an augmented reality experience associated with the target word.

18. The method of claim 17 , wherein the LM score of the target n-gram is returned to an original value after the augmented reality experience is terminated.

19. A system comprising:

at least one processor configured to perform operations comprising:

accessing, by an automatic speech recognition (ASR) engine, a language model (LM) that includes a plurality of n-grams, each of the plurality of n-grams comprising a respective sequence of words and corresponding LM score, the ASR engine comprising an acoustic component and a decoder;

converting, by the acoustic component, voice input into logits of inferred characters of phonemes;

selecting a target word to boost in the LM;

receiving input comprising a boosting factor including a specific boosting value for the target word;

identifying a target n-gram in the LM that includes the target word;

identifying a subset of n-grams of the plurality of n-grams that includes words in a portion of the target n-gram excluding the target word;

within the subset of n-grams that exclude the target word, identifying an individual n-gram having an individual LM score that corresponds to the specific boosting value of the boosting factor received in the input;

adjusting the LM score of the target n-gram using the individual LM score of the individual n-gram, within the subset of n-grams that exclude the target word, that corresponds to the specific boosting value of the boosting factor received in the input; and

processing, by the decoder, the logits of inferred characters of phonemes using the LM comprising the adjusted LM score of the target n-gram to generate a transcription of the voice input.

20. A non-transitory machine-readable storage medium that includes instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

accessing, by an automatic speech recognition (ASR) engine, a language model (LM) that includes a plurality of n-grams, each of the plurality of n-grams comprising a respective sequence of words and corresponding LM score, the ASR engine comprising an acoustic component and a decoder;

converting, by the acoustic component, voice input into logits of inferred characters of phonemes;

selecting a target word to boost in the LM;

receiving input comprising a boosting factor including a specific boosting value for the target word;

identifying a target n-gram in the LM that includes the target word;

identifying a subset of n-grams of the plurality of n-grams that includes words in a portion of the target n-gram excluding the target word;

within the subset of n-grams that exclude the target word, identifying an individual n-gram having an individual LM score that corresponds to the specific boosting value of the boosting factor received in the input;

adjusting the LM score of the target n-gram using the individual LM score of the individual n-gram, within the subset of n-grams that exclude the target word, that corresponds to the specific boosting value of the boosting factor received in the input; and

processing, by the decoder, the logits of inferred characters of phonemes using the LM comprising the adjusted LM score of the target n-gram to generate a transcription of the voice input.

Assignments (2)

Continuity (1)

References Cited (400)