Training data sequence for RNN-T based global English model
View Patent ↗A computer-implemented method for preparing training data for a speech recognition model is provided including obtaining a plurality of audio data sets, each audio data set having a different acoustic feature and sorting sentences from the plurality of audio data sets so that similar sentences from different audio data sets are positioned closely, while imposing a weak constraint on audio length, to train the speech recognition model.
1. A computer-implemented method for preparing training data for a speech recognition model, the method comprising:
obtaining a plurality of audio data sets, each audio data set having a different acoustic feature; and
training a recurrent neural network transducer speech recognition model by sorting sentences from the plurality of audio data sets so that similar sentences from different audio data sets are positioned closely as a primary constraint by utilizing a similarity-score dependent penalty imposed for composed dissimilar data based on distances between sentences on a word vector and at least two hyperparameters, while imposing a secondary constraint on audio length by comparing audio distances between the sentences from utterances extracted from the plurality of audio data sets.
2. The computer-implemented method of claim 1 , further comprising sampling the plurality of audio data sets from data pools each having the different acoustic feature so that the sampled audio data sets include a plurality of sets of similar sentences.
3. The computer-implemented method of claim 2 , wherein a score penalty is presented to control a variety of sentences.
4. The computer-implemented method of claim 1 , wherein the similar sentences are similar sentences with different dialects of a target language.
5. The computer-implemented method of claim 4 , wherein the speech recognition model is a global speech recognition model for the target language.
6. The computer-implemented method of claim 1 , wherein the similar sentences from the different audio data sets are grouped into mini-batches.
7. The computer-implemented method of claim 6 , wherein each mini-batch of the mini-batches includes sentence pairs between different English dialects.
8. The computer-implemented method of claim 6 , wherein each mini-batch of the mini-batches includes a similar amount of dialect data.
9. The computer-implemented method of claim 1 , wherein the similarity between different English dialects of the similar sentences from different audio data sets is given by:
argmax
j
F
(
S
i
BSL
,
j
n
)
-
P
(
F
(
S
i
BSL
,
j
n
)
)
where F(a, b) is a distance between sentences a and b based on a word vector of n-word sequences and P(d) is a similarity-score dependent penalty not to compose biased text data, where a=S i BSL ; b=j n ; and d=F(S i BSL , j n ).
10. The computer-implemented method of claim 9 , wherein the similarity-score dependent penalty is given by:
P ( d )=γ e κd −γ( d> 0)
where γ, κ are hyper parameters.
11. A computer program product for preparing training data for a speech recognition model, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
obtain a plurality of audio data sets, each audio data set having a different acoustic feature; and
train a recurrent neural network transducer speech recognition model by sorting sentences from the plurality of audio data sets so that similar sentences from different audio data sets are positioned closely as a primary constraint by utilizing a similarity-score dependent penalty imposed for composed dissimilar data based on distances between sentences on a word vector and at least two hyperparameters, while imposing a secondary constraint on audio length by comparing audio distances between the sentences from utterances extracted from the plurality of audio data sets.
12. The computer program product of claim 11 , wherein the plurality of audio data sets are sampled from data pools each having the different acoustic feature so that the sampled audio data sets include a plurality of sets of similar sentences.
13. The computer program product of claim 12 , wherein a score penalty is presented to control a variety of sentences.
14. The computer program product of claim 11 , wherein the similar sentences are similar sentences with different dialects of a target language.
15. The computer program product of claim 14 , wherein the speech recognition model is a global speech recognition model for the target language.
16. The computer program product of claim 11 , wherein the similar sentences from the different audio data sets are grouped into mini-batches.
17. The computer program product of claim 16 , wherein each mini-batch of the mini-batches includes sentence pairs between different English dialects.
18. The computer program product of claim 16 , wherein each mini-batch of the mini-batches includes a similar amount of dialect data.
19. The computer program product of claim 11 , wherein the similarity between different English dialects of the similar sentences from different audio data sets is given by:
argmax
j
F
(
S
i
BSL
,
j
n
)
-
P
(
F
(
S
i
BSL
,
j
n
)
)
where F(a, b) is a distance between sentences a and b based on a word vector of n-word sequences and P(d) is a similarity-score dependent penalty not to compose biased text data, where a=S i BSL ; b=j n ; and d=F(S i BSL , j n ).
20. The computer program product of claim 19 , wherein the similarity-score dependent penalty is given by:
P ( d )=γ e κd −γ( d> 0)
where γ, κ are hyper parameters.
21. A system for preparing training data for a speech recognition model, the system comprising:
a memory; and
one or more processors in communication with the memory configured to:
obtain a plurality of audio data sets, each audio data set having a different acoustic feature; and
train a recurrent neural network transducer speech recognition model by sorting sentences from the plurality of audio data sets so that similar sentences from different audio data sets are positioned closely as a primary constraint by utilizing a similarity-score dependent penalty imposed for composed dissimilar data based on distances between sentences on a word vector and at least two hyperparameters, while imposing a secondary constraint on audio length by comparing audio distances between the sentences from utterances extracted from the plurality of audio data sets.
22. The system of claim 21 , wherein a similarity between different English dialects of the similar sentences from different audio data sets is given by:
argmax
j
F
(
S
i
BSL
,
j
n
)
-
P
(
F
(
S
i
BSL
,
j
n
)
)
where F(a, b) is a distance between sentences a and b based on a word vector of n-word sequences and P(d) is the similarity-score dependent penalty not to compose biased text data, where a=S i BSL ; b=j n ; and d=F(S i BSL , j n ).
23. A computer-implemented method for preparing training data for a speech recognition model, the method comprising:
obtaining a plurality of audio data sets, each audio data set having a different acoustic feature;
training a recurrent neural network transducer speech recognition model by sorting sentences from the plurality of audio data sets so that similar sentences from different audio data sets are positioned closely as a primary constraint by comparing audio distances between the sentences from utterances extracted from the plurality of audio data sets and utilizing a similarity-score dependent penalty imposed for composed dissimilar data based on distances between sentences on a word vector and at least two hyperparameters; and
grouping the similar sentences from the different audio data sets into mini-batches, wherein each mini-batch of the mini-batches includes sentence pairs between different English dialects.
24. The computer-implemented method of claim 23 , wherein a similarity between different English dialects of the similar sentences from different audio data sets is given by:
argmax
j
F
(
S
i
BSL
,
j
n
)
-
P
(
F
(
S
i
BSL
,
j
n
)
)
where F(a, b) is a distance between sentences a and b based on a word vector of n-word sequences and P(d) is the similarity-score dependent penalty not to compose biased text data, where a=S i BSL ; b=j n ; and d=F(S i BSL , j n ).
25. A computer program product for preparing training data for a speech recognition model, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
obtain a plurality of audio data sets, each audio data set having a different acoustic feature;
train a recurrent neural network transducer speech recognition model by sorting sentences from the plurality of audio data sets so that similar sentences from different audio data sets are positioned closely as a primary constraint by comparing audio distances between the sentences from utterances extracted from the plurality of audio data sets and utilizing a similarity-score dependent penalty imposed for composed dissimilar data based on distances between sentences on a word vector and at least two hyperparameters; and
group the similar sentences from the different audio data sets into mini-batches, wherein each mini-batch of the mini-batches includes sentence pairs between different English dialects.