Method and system for automatic speech recognition (ASR) using multi-task learned (MTL) embeddings
State of the art Acoustic Models (AM), which are trained using data from one environment, may fail to adapt to another environment, and as a result, application is restricted. The disclosure herein generally relates to speech signal processing, and, more particularly, to a method and system for Automatic Speech Recognition (ASR) using Multi-task Learned Embeddings (MTL). In this approach, MTL embeddings are extracted from an MTL neural network that has been trained using feature vectors from a plurality of speech files. The MTL embeddings are then used for generating an acoustic model, which maybe then used for the purpose of Automatic Speech Recognition, along with the feature vectors and the MTL embeddings.
1 . A processor implemented method, comprising:
receiving, via one or more hardware processors, a plurality of speech files signals as input data;
extracting, via the one or more hardware processors, a plurality of feature vectors from the input data;
training, via the one or more hardware processors, a multi-task learning (MTL) neural network, using one or more of the plurality of feature vectors, wherein the trained MTL neural network able to perform: process the speech signals and extract one or more information from the speech signals, classify a room in which one or more speakers are present from whom the speech signals are being received;
extracting, via the one or more hardware processors, MTL embeddings from the MTL neural network, wherein extracted MTL embeddings are encoded with a plurality of environmental parameters and a plurality of speaker parameters, wherein the plurality of environmental parameters comprises information on classification of the room as one of large, small and medium as identified by the MTL neural network, and a noise information comprising a Signal to Noise Ratio (SNR), wherein the MTL embeddings are extracted by: initially performing frame-level features extraction using a plurality of Time delay neural network (TDNN) layers, further output of the plurality of TDNN layers is subjected to statistical pooling before performing an utterance level feature extraction and output of the utterance level feature extraction is processed using a plurality of softmax layers to obtain a speaker label and a room size label;
generating, via the one or more hardware processors, an acoustic model by training a deep neural network with a multi-task loss function, using the feature vectors and the MTL embeddings; and
decoding a transcript for an utterance from the acoustic model using the feature vectors extracted from the plurality of speech signals and the MTL embeddings.
2 . The processor implemented method of claim 1 , wherein the input data is received using an interface or received in real-time as speech signals from an environment using sensors.
3 . The processor implemented method of claim 1 , wherein the plurality of speaker parameters comprise speaker identity, gender, and age.
4 . A system, comprising:
one or more hardware processors;
a communication interface; and
a memory storing a plurality of instructions, wherein the plurality of instructions when executed, cause the one or more hardware processors to:
receive a plurality of speech signals as input data;
extract a plurality of feature vectors from the input data;
train a multi-task learning (MTL) neural network, using one or more of the plurality of feature vectors, wherein the trained MTL neural network able to perform: process the speech signals and extract one or more information from the speech signals, classify a room in which one or more speakers are present from whom the speech signals are being received;
extract MTL embeddings from the MTL neural network, wherein extracted MTL embeddings are encoded with a plurality of environmental parameters and a plurality of speaker parameters, wherein the plurality of environmental parameters comprises information on classification of the room as one of large, small and medium as identified by the MTL neural network, and a noise information comprising a Signal to Noise Ratio (SNR), wherein the MTL embeddings are extracted by: initially performing frame-level features extraction using a plurality of Time delay neural network (TDNN) layers, further output of the plurality of TDNN layers is subjected to statistical pooling before performing an utterance level feature extraction and output of the utterance level feature extraction is processed using a plurality of softmax layers to obtain a speaker label and a room size label;
generate an acoustic model by training a deep neural network with a multi-task loss function, using the feature vectors and the MTL embeddings; and
decoding a transcript for an utterance from the acoustic model using the feature vectors extracted from the plurality of speech signals and the MTL embeddings.
5 . The system of claim 4 , wherein the one or more hardware processors are further configured to receive the input data using an interface or received in real-time as speech signals from an environment using sensors.
6 . The system of claim 4 , wherein the one or more hardware processors are configured to obtain information on speaker identity, gender, and age as the plurality of speaker parameters.
7 . One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
receiving a plurality of speech signals as input data;
extracting a plurality of feature vectors from the input data;
training a multi-task learning (MTL) neural network, using one or more of the plurality of feature vectors, wherein the trained MTL neural network able to perform: process the speech signals and extract one or more information from the speech signals, classify a room in which one or more speakers are present from whom the speech signals are being received;
extracting MTL embeddings from the MTL neural network, wherein extracted MTL embeddings are encoded with a plurality of environmental parameters and a plurality of speaker parameters, wherein the plurality of environmental parameters comprises information on classification of the room as one of large, small and medium as identified by the MTL neural network, and a noise information comprising a Signal to Noise Ratio (SNR), wherein the MTL embeddings are extracted by: initially performing frame-level features extraction using a plurality of Time delay neural network (TDNN) layers, further output of the plurality of TDNN layers is subjected to statistical pooling before performing an utterance level feature extraction and output of the utterance level feature extraction is processed using a plurality of softmax layers to obtain a speaker label and a room size label;
generating an acoustic model by training a deep neural network with a multi-task loss function, using the feature vectors and the MTL embeddings; and
decoding a transcript for an utterance from the acoustic model using the feature vectors extracted from the plurality of speech signals and the MTL embeddings.
8 . The one or more non-transitory machine-readable information storage mediums of claim 7 , further comprising receiving the input data using an interface or received in real-time as speech signals from an environment using sensors.
9 . The one or more non-transitory machine-readable information storage mediums of claim 7 , wherein the plurality of speaker parameters comprise speaker identity, gender, and age.