Language identification classifier trained using encoded audio from encoder of pre-trained speech-to-text system
View Patent ↗An example system includes a processor to receive encoded audio from an encoder of a pre-trained speech-to-text (STT) model. The processor is to further train a language identification (LID) classifier to detect a language of the encoded audio using training samples labeled by language.
1. A system, comprising a processor to:
receive an encoded audio from an encoder of a pre-trained speech-to-text (STT) model;
train a language identification (LID) classifier to detect a language of the encoded audio using training samples labeled by language;
and the system further comprising the processor to:
receive an audio sample to be converted into text;
encode the audio sample into a second encoded audio;
classify the second encoded audio via the trained LID classifier; and
generate, in response to detecting that the second encoded audio is classified as a target language, text in the target language based on the second encoded audio and prediction from a predictor of the pre-trained STT model;
wherein the LID classifier includes weights and linear projections being multiplied in a multiplier; and
wherein the LID classifier further includes an average pooling layer as a multi-head weighted-average pooling layer.
2. The system of claim 1 , wherein the encoder comprises a recurrent neural network transducer (RNN-T) encoder.
3. The system of claim 1 , wherein the encoder is pre-trained on one language.
4. The system of claim 1 , wherein the STT model comprises a plurality of predictors dedicated to different languages, wherein the LID classifier is to classify a second encoded audio corresponding to an audio sample to be converted into text and select a corresponding dedicated predictor based on the classification.
5. The system of claim 1 , wherein the encoder of the STT model is pre-trained with the plurality of predictors for different languages.
6. The system of claim 1 , wherein the encoded audio comprises a frame-level feature vector.
7. A computer-implemented method, comprising:
receiving, via a processor, an encoded audio from an encoder of a pre-trained speech-to-text (STT) model;
training, via the processor, a language identification (LID) classifier to detect a language of the encoded audio using training samples labeled by language;
the computer-implemented method further comprising:
receiving, via the processor, an audio sample to be converted into text;
encoding, via the processor, the audio sample into a second encoded audio;
classifying, via the processor, the second encoded audio via the trained LID classifier; and
sending, via the processor, the second encoded audio to a dedicated predictor of a plurality of predictors dedicated to different languages based on the classification;
wherein the LID classifier includes weights and linear projections being multiplied in a multiplier; and
wherein the LID classifier further includes an average pooling layer as a multi-head weighted-average pooling layer.
8. The computer-implemented method of claim 7 , comprising:
receiving, via the processor, an audio sample to be converted into text;
encoding, via the processor, the audio sample into a second encoded audio; and
classifying, via the processor, the second encoded audio via the trained LID classifier.
9. The computer-implemented method of claim 8 , comprising stopping, via the processor, processing of the audio sample in response to detecting that the second encoded audio is not classified as a target language.
10. The computer-implemented method of claim 8 , comprising generating, via the processor, text in the target language based on the second encoded audio and prediction from a predictor of the pre-trained STT model in response to detecting that the second encoded audio is classified as a target language.
11. The computer-implemented method of claim 7 , comprising generating the text from the second encoded audio via the dedicated predictor.
12. The computer-implemented method of claim 7 , wherein classifying the second encoded audio comprises applying a softmax function to linear projections of pooled weighted averages and classifying the second encoded audio based on a language class with a highest decimal probability.
13. A computer program product for training language identification classifiers, the computer program product comprising a computer-readable storage medium having program code embodied therewith, the program code executable by a processor to cause the processor to:
receive encoded audio from an encoder of a pre-trained speech-to-text (STT) model;
train a language identification (LID) classifier to detect a language of the encoded audio using training samples labeled by language;
the computer program product further comprising program code executable by the processor to:
receive an audio sample to be converted into text;
encode the audio sample into a second encoded audio;
classify the second encoded audio via the trained LID classifier;
send the second encoded audio to a dedicated predictor of a plurality of predictors dedicated to different languages based on the classification; and
generate the text from the encoded audio via the dedicated predictor;
wherein the LID classifier includes weights and linear projections being multiplied in a multiplier; and
wherein the LID classifier further includes an average pooling layer as a multi-head weighted-average pooling layer.
14. The computer program product of claim 13 , further comprising program code executable by the processor to:
receive an audio sample to be converted into text;
encode the audio sample into a second encoded audio; and
classify the second encoded audio via the trained LID classifier.
15. The computer program product of claim 14 , further comprising program code executable by the processor to stop processing of the audio sample in response to detecting that the encoded audio is not classified as a target language.
16. The computer program product of claim 14 , further comprising program code executable by the processor to generate text in the target language based on the second encoded audio and prediction from a predictor of the pre-trained STT model in response to detecting that the second encoded audio is classified as a target language.
17. The computer program product of claim 13 , further comprising program code executable by the processor to apply a softmax function to linear projections of a pooled weighted averages and classify the second encoded audio based on a language class with a highest decimal probability.