Granted Patent B2

US 12,367,343 · App. 17/946,389 · Granted Jul 22, 2025

Inventors: Prakash Ranganathan (Villupuram, IN); Saurabh Tahiliani (Noida, IN)

Assignee: Verizon Patent and Licensing Inc.

G06F40/284G06F40/232G06F40/40

Abstract

Claims (105)

1. A method performed by a transcription system, the method comprising:

receiving audio data from one or more devices;

performing one or more pre-processing operations on the audio data to generate pre-processed audio data, wherein the one or more pre-processing operations include one or more of an adjustment of an amplitude of the audio data, an adjustment of a frequency of the audio data, or a removal of noise from the audio data;

generating a first transcript based on the pre-processed audio data;

determining, using a first machine learning model of the transcription system, that a portion of the first transcript is incorrect,

wherein the first machine learning model is configured to detect incorrect portions of the transcripts;

generating, using a second machine learning model, additional data for transcribing the pre-processed audio data based on determining that the portion of the first transcript is incorrect,

wherein the additional data is generated using a portion of the pre-processed audio data corresponding to the portion of the first transcript;

generating a second transcript based on the pre-processed audio data and the additional data; and

providing the second transcript.

2. The method of claim 1 , wherein generating the additional data comprises:

determining the portion of the pre-processed audio data corresponding to the portion of the first transcript; and

generating an alternative transcript of the portion of the pre-processed audio data based on determining that the portion of the first transcript is incorrect,

wherein the additional data includes the portion of the pre-processed audio data and the alternative transcript of the portion of the pre-processed audio data.

3. The method of claim 2 , further comprising:

training a decoder to detect incorrect transcripts,

wherein the decoder is configured to determine that the portion of the first transcript is incorrect; and

training an encoder to generate correct transcripts,

wherein the encoder is configured to generate the alternative transcript of the portion.

4. The method of claim 2 , further comprising:

providing, to an encoder, training data that includes a plurality of tokens,

wherein one or more first tokens, of the plurality of tokens, are masked, and

wherein one or more second tokens, of the plurality of tokens, are not masked; and

training the encoder to predict one or more values for the one or more first tokens,

wherein the encoder is configured to generate the alternative transcript of the portion.

5. The method of claim 4 , further comprising:

providing, to a decoder, the one or more values and the one or more second tokens; and

training the decoder to determine that the one or more values were incorrectly predicted by the encoder or were correctly predicted by the encoder,

wherein the decoder is configured to determine that the portion of the first transcript is incorrect.

6. The method of claim 4 , wherein training the encoder comprises:

training the encoder to predict one or more values for the one or more first tokens based on at least one of:

determining a relationship between the one or more first tokens and the one or more second tokens,

determining a context associated with the one or more first tokens and the one or more second tokens, or

determining a meaning associated with the one or more first tokens and the one or more second tokens.

7. The method of claim 1 , further comprising:

determining whether the second transcript includes a portion that is incorrect; and

providing the second transcript to the one or more devices based on the second transcript not including a portion that is incorrect.

8. A device, comprising:

one or more processors configured to:

receive audio data from one or more devices;

perform one or more preprocessing operation on the audio data to generate pre-processed audio data, wherein the one or more pre-processing operations include one or more of an adjustment of an amplitude of the audio data, an adjustment of a frequency of the audio data, or a removal of noise from the audio data;

generate a first transcript based on the pre-processed audio data;

determine, using a first machine learning model of the device, that a portion of the first transcript is incorrect;

generate, using a second machine learning model, additional data for transcribing the pre-processed audio data based on determining that the portion of the first transcript is incorrect,

wherein the additional data is generated using a portion of the pre-processed audio data corresponding to the portion of the first transcript;

generate a second transcript based on the pre-processed audio data and the additional data;

determine whether the second transcript includes a portion that is incorrect; and

selectively provide the second transcript or generate a third transcript of the pre-processed audio data based on determining whether the second transcript includes the portion that is incorrect,

wherein the second transcript is provided based on the second transcript not including the portion that is incorrect, and

wherein the third transcript is generated based on the second transcript including the portion that is incorrect.

9. The device of claim 8 , wherein the additional data is first additional data, and

wherein the one or more processors, to selectively provide the second transcript or generate the third transcript, are configured to:

determine that the second transcript includes a portion that is incorrect;

determine a portion of the pre-processed audio data corresponding to the portion of the second transcript;

generate second additional data for transcribing the pre-processed audio data based on determining that the portion of the second transcript is incorrect,

wherein the second additional data is generated using a portion of the pre-processed audio data corresponding to the portion of the second transcript; and

generate the third transcript based on the pre-processed audio data and the second additional data.

10. The device of claim 8 , wherein, to remove the noise, the one or more processors are configured to:

remove white noise from the audio data.

11. The device of claim 8 , wherein the first transcript includes a plurality of tokens, and

wherein the one or more processors, to determine that the portion of the first transcript is incorrect, are configured to:

determine that one or more tokens, of the plurality of tokens, are incorrect.

12. The device of claim 8 , wherein the one or more processors, to generate the additional data, are configured to:

generate an alternative transcript of the portion of the pre-processed audio data based on determining that the portion of the first transcript is incorrect,

wherein the additional data includes the portion of the pre-processed audio data and the alternative transcript of the portion of the pre-processed audio data.

13. The device of claim 12 , wherein the one or more processors, to determine that the portion of the first transcript is incorrect, are configured to:

determine, using a decoder, that the portion of the first transcript is incorrect; and

wherein the one or more processors, to generate the alternative transcript, are configured to:

generate, using an encoder, the alternative transcript.

14. The device of claim 13 , wherein the one or more processors are further configured to:

train the decoder to detect incorrect transcripts; and

train the encoder to generate correct transcripts.

15. A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:

one or more instructions that, when executed by one or more processors of a device, cause the device to:

receive audio data from one or more devices;

perform, prior to a transcription of the audio data, one or more pre-processing operations on the audio data to generate pre-processed audio data, wherein the one or more pre-processing operations include one or more of an adjustment of an amplitude of the audio data, an adjustment of a frequency of the audio data, or a removal of noise from the audio data;

determine, using a first machine learning model, that a portion of a first transcript of the pre-processed audio data is incorrect;

generate, using a machine learning model, additional data for transcribing the pre-processed audio data based on determining that the portion of the first transcript is incorrect,

wherein the additional data is generated using a portion of the pre-processed audio data corresponding to the portion of the first transcript;

generate a second transcript based on the pre-processed audio data and the additional data;

determine whether the second transcript includes a portion that is incorrect; and

selectively provide the second transcript or generate a third transcript of the pre-processed audio data based on determining whether the second transcript includes the portion that is incorrect,

wherein the second transcript is provided based on the second transcript not including the portion that is incorrect, and

wherein the third transcript is generated based on the second transcript including the portion that is incorrect.

16. The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, when executed by the one or more processors, further cause the device to:

generate, using the first machine learning model, the first transcript based on the pre-processed audio data.

17. The non-transitory computer-readable medium of claim 15 , wherein the additional data is first additional data,

wherein the one or more instructions, that cause the device to selectively provide the second transcript or generate the third transcript, cause the device to:

determine that the second transcript includes a portion that is incorrect;

determine a portion of the pre-processed audio data corresponding to the portion of the second transcript;

generate second additional data for transcribing the pre-processed audio data based on determining that the portion of the second transcript is incorrect,

wherein the second additional data is generated using a portion of the pre-processed audio data corresponding to the portion of the second transcript; and

generate the third transcript based on the pre-processed audio data, the portion of the pre-processed audio data, and the second additional data.

18. The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, that cause the device to determine that the portion of the first transcript is incorrect, cause the device to:

determine, using a decoder of the device, that the portion of the first transcript is incorrect; and

wherein the one or more processors, to generate the additional data, are configured to:

generate, using an encoder of the device, the additional data.

19. The non-transitory computer-readable medium of claim 18 , wherein the one or more instructions further cause the device to:

train the decoder to detect incorrect transcripts; and

train the encoder to generate correct transcripts.

20. The non-transitory computer-readable medium of claim 19 , wherein the one or more instructions to train the encoder further cause the device to:

train the encoder to predict one or more values for tokens based on at least one of:

a relationship between tokens included in training data,

a context associated with the tokens, or

a meaning associated with the tokens.

Assignments (1)

Continuity (1)

References Cited (42)