Transcription generation

1 . A method of transcript generation, the method comprising:

obtaining, from a device, audio data of a communication session between the device and another device;

directing the audio data to an automatic speech recognition system configured to generate transcript data that includes a transcription of the audio data, the generating of the transcript data including:

extracting features from the audio data to generate a vector of features;

determining a set of probabilities using the vector of features;

determining one or more hypothesized words based on the set of probabilities; and

outputting the transcription of the audio data based on the hypothesized words;

directing the audio data to one or more secondary automatic speech recognition systems each configured to generate secondary transcript data that includes a separate transcription of the audio data, the generating of the secondary transcript data including:

extracting features from the audio data to generate a vector of features;

determining a set of probabilities using the vector of features;

determining one or more hypothesized words based on the set of probabilities; and

outputting the separate transcription of the audio data based on the one or more hypothesized words;

providing the transcript data from the automatic speech recognition system and the secondary transcript data from the secondary automatic speech recognition systems to a large language model for the large language model to correct errors in at least one of the transcript data or the secondary transcript data;

in response to providing the transcript data and the secondary transcript data to the large language model, obtaining correction data generated by the large language model using the transcript data and the secondary transcript data, wherein the correction data includes correction of errors in at least one of the transcript data or the secondary transcript data; and

directing, to the device, data based on the correction data.

2 . The method of claim 1 , wherein the transcript data includes one or more words included in the transcription of the audio data and a transcription score generated by the automatic speech recognition system indicates how well words in the transcript data match words in the audio data, the large language model being directed to use the transcription score to correct errors in the one or more words included in the transcription.

3 . The method of claim 1 , wherein the transcript data and the secondary transcript data includes transcription scores generated by the automatic speech recognition system and the secondary automatic speech recognition systems, the transcription scores indicating how well words in the transcript data match words in the audio data, the large language model being directed to use the transcription scores to generate the correction data.

4 . The method of claim 1 , further comprising before obtaining the correction data from the large language model, providing the transcript data to the device, wherein the correction data includes revisions to transcript data previously obtained by the device.

5 . The method of claim 1 , further comprising providing a prompt to the large language model, the prompt including directions for the large language model to process the transcript data.

6 . The method of claim 1 , wherein the transcript data is derived from a first portion of the audio data of the communication session, the method further comprising providing context to the large language model, wherein the context includes transcript data derived from a second portion of the audio data that occurs before the first portion during the communication session.

7 . The method of claim 1 , wherein the automatic speech recognition system includes an acoustic model, and the large language model uses a transformer architecture.

8 . The method of claim 1 , further comprising:

combining the secondary transcript data and the transcript data, the combined transcript data being provided to the large language model for the large language model to correct errors in the combined transcript data, the transcript data provided to the large language model as part of the combined transcript data.

9 . The method of claim 8 , wherein the combined transcript data includes a combined transcription score generated based on transcription scores generated by the automatic speech recognition system and the secondary automatic speech recognition systems, the combined transcription score indicating how well words in the combined transcript data match words in the audio data, the large language model being directed to use the combined transcription score to generate the correction data.

10 . The method of claim 1 , wherein the transcript data provided to the large language model includes a word lattice, a word confusion network, or a n-best list used to generate the transcription of the audio data.

11 . At least one non-transitory computer-readable media configured to store one or more instructions that, in response to being executed by a system, cause or direct the system to perform the method of claim 1 .

12 . A system comprising: one or more computer readable mediums including instructions;

one or more computing systems coupled to the one or more computer readable mediums and configured to execute the instructions to cause or direct the system to perform operations, the operations comprising:

obtaining, from a device, audio data of a communication session between the device and another device;

directing the audio data to an automatic speech recognition system configured to generate transcript data that includes a transcription of the audio data, the generating of the transcript data including:

extracting features from the audio data to generate a vector of features;

determining a set of probabilities using the vector of features;

determining one or more hypothesized words based on the set of probabilities; and

outputting the transcription of the audio data based on the hypothesized words;

directing the audio data to one or more secondary automatic speech recognition systems each configured to generate secondary transcript data that includes a separate transcription of the audio data, the generating of the secondary transcript data including:

extracting features from the audio data to generate a vector of features;

determining a set of probabilities using the vector of features;

determining one or more hypothesized words based on the set of probabilities; and

outputting the separate transcription of the audio data based on the one or more hypothesized words;

providing the transcript data from the automatic speech recognition system and the secondary transcript data from the secondary automatic speech recognition systems to a large language model for the large language model to correct errors in at least one of the transcript data or the secondary transcript data;

in response to providing the transcript data and the secondary transcript data to the large language model, obtaining correction data generated by the large language model using the transcript data and the secondary transcript data, wherein the correction data includes correction of errors in at least one of the transcript data or the secondary transcript data; and

directing, to the device, data based on the correction data.

13 . The system of claim 12 , wherein the transcript data includes one or more words included in the transcription of the audio data and a transcription score generated by the automatic speech recognition system, the transcription scores indicating how well words in the transcript data match words in the audio data, the large language model being directed to use the transcription score to correct errors in the one or more words included in the transcription.

14 . The system of claim 12 , wherein the transcript data and the secondary transcript data includes transcription scores generated by the automatic speech recognition system and the secondary automatic speech recognition systems, the transcription scores indicating how well words in the transcript data match words in the audio data, the large language model being directed to use the transcription scores to generate the correction data.

15 . The system of claim 12 , wherein the transcript data is derived from a first portion of the audio data of the communication session, the operations further comprise providing context to the large language model, wherein the context includes transcript data derived from a second portion of the audio data that occurs before the first portion during the communication session.

16 . The system of claim 12 , wherein the automatic speech recognition system includes an acoustic model, and the large language model uses a transformer architecture.

17 . The system of claim 12 , wherein the operations further comprise:

combining the secondary transcript data and the transcript data, the combined transcript data being provided to the large language model for the large language model to correct errors in the combined transcript data, the transcript data provided to the large language model as part of the combined transcript data.

18 . The system of claim 17 , wherein the combined transcript data includes a combined transcription score generated based on transcription scores generated by the automatic speech recognition system and the secondary automatic speech recognition systems, the combined transcription score indicating how well words in the combined transcript data match words in the audio data, the large language model being directed to use the combined transcription score to generate the correction data.

19 . The system of claim 12 , wherein the transcript data provided to the large language model is a transcription of a portion of the audio data, a number of words in the portion of the audio data based on one or more parameters of the large language model.

20 . A method of transcript generation, the method comprising:

obtaining audio data;

directing the audio data to an automatic speech recognition system configured to generate transcript data that includes a transcription of the audio data, the generating of the transcript data including:

extracting features from the audio data to generate a vector of features;

determining a set of probabilities using the vector of features;

determining one or more hypothesized words based on the set of probabilities; and

outputting the transcription of the audio data based on the hypothesized words;

directing the audio data to one or more secondary automatic speech recognition systems each configured to generate secondary transcript data that includes a separate transcription of the audio data, the generating of the secondary transcript data including:

extracting features from the audio data to generate a vector of features;

determining a set of probabilities using the vector of features;

determining one or more hypothesized words based on the set of probabilities; and

outputting the separate transcription of the audio data based on the one or more hypothesized words;

providing the transcript data from the automatic speech recognition system and the secondary transcript data from the secondary automatic speech recognition systems to a large language model for the large language model to correct errors in at least one of the transcript data or the secondary transcript data;

in response to providing the transcript data and the secondary transcript data to the large language model, obtaining, correction data generated by the large language model using the transcript data and the secondary transcript data, wherein the correction data includes correction of errors in at least one of the transcript data or the secondary transcript data; and

generating revised transcript data using the correction data.