Data sorting for generating RNN-T models
View Patent ↗A computer-implemented method for preparing training data for a speech recognition model is provided including obtaining a plurality of sentences from a corpus, dividing each phoneme in each sentence of the plurality of sentences into three hidden states, calculating, for each sentence of the plurality of sentences, a score based on a variation in duration of the three hidden states of each phoneme in the sentence, and sorting the plurality of sentences by using the calculated scores.
1. A computer-implemented method for preparing training data for a speech recognition model, the method comprising:
obtaining a plurality of sentences from a corpus;
dividing each phoneme in each sentence of the plurality of sentences into three hidden states;
calculating, for each sentence of the plurality of sentences, a score based on a variation in duration of the three hidden states of each phoneme in the sentence; and
sorting the plurality of sentences by using the calculated scores.
2. The computer-implemented method of claim 1 , wherein each score is calculated by using an average duration of the three hidden states in each phoneme in the sentence.
3. The computer-implemented method of claim 1 , wherein each score is calculated by using a duration for each of the three hidden states in each phoneme in the sentence previously estimated from clean speech.
4. The computer-implemented method of claim 1 , wherein the score is a phoneme-independent evenness score when statistics from clean speech are not available.
5. The computer-implemented method of claim 4 , wherein the phoneme-independent evenness score is given as:
S
=
1
N
∑
n
=
0
N
∑
x
=
{
b
,
m
,
e
}
(
d
n
,
x
-
μ
n
)
2
where d n,x is a duration of an x-th state in an n-th phoneme such that x takes either beginning (b) or middle (m) or ending (e), μ n is an average duration of the three hidden states in the n-th phoneme obtained by d n,x , and N is a total number of phonemes in the sentence.
6. The computer-implemented method of claim 1 , wherein the score is a phoneme-dependent evenness score when statistics from clean speech are available.
7. The computer-implemented method of claim 6 , wherein the phoneme-dependent evenness score is given as:
S
=
1
N
∑
n
=
0
N
∑
x
=
{
b
,
m
,
e
}
(
d
n
,
x
-
λ
k
(
n
)
,
x
)
2
where λ k(n),x is an ideal duration for an x-th state of phoneme k(n) to which an n-th phoneme should be referred, estimated from the clean speech in advance and k(n) is a function to map the n-th phoneme to a corresponding phoneme.
8. A computer program product for preparing training data for a speech recognition model including a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to: obtain a plurality of sentences from a corpus; divide each phoneme in each sentence of the plurality of sentences into three hidden states; calculate, for each sentence of the plurality of sentences, a score based on a variation in duration of the three hidden states of each phoneme in the sentence; and sort the plurality of sentences by using the calculated scores.
9. The computer program product of claim 8 , wherein each score is calculated by using an average duration of the three hidden states in each phoneme in the sentence.
10. The computer program product of claim 8 , wherein each score is calculated by using a duration for each of the three hidden states in each phoneme in the sentence previously estimated from clean speech.
11. The computer program product of claim 8 , wherein the score is a phoneme-independent evenness score when statistics from clean speech are not available.
12. The computer program product of claim 11 , wherein the phoneme-independent evenness score is given as:
S
=
1
N
∑
n
=
0
N
∑
x
=
{
b
,
m
,
e
}
(
d
n
,
x
-
μ
n
)
2
where d n,x is a duration of an x-th state in an n-th phoneme such that x takes either beginning (b) or middle (m) or ending (e), μ n is an average duration of the three hidden states in the n-th phoneme obtained by d n,x , and N is a total number of phonemes in the sentence.
13. The computer program product of claim 8 , wherein the score is a phoneme-dependent evenness score when statistics from clean speech are available.
14. The computer program product of claim 13 , wherein the phoneme-dependent evenness score is given as:
S
=
1
N
∑
n
=
0
N
∑
x
=
{
b
,
m
,
e
}
(
d
n
,
x
-
λ
k
(
n
)
,
x
)
2
where λ k(n),x is an ideal duration for an x-th state of phoneme k(n) to which an n-th phoneme should be referred, estimated from the clean speech in advance and k(n) is a function to map the n-th phoneme to a corresponding phoneme.
15. A system for preparing training data for a speech recognition model, the system comprising:
a memory; and
one or more processors in communication with the memory configured to:
obtain a plurality of sentences from a corpus;
divide each phoneme in each sentence of the plurality of sentences into three hidden states;
calculate, for each sentence of the plurality of sentences, a score based on a variation in duration of the three hidden states of each phoneme in the sentence; and
sort the plurality of sentences by using the calculated scores.
16. The system of claim 15 , wherein each score is calculated by using an average duration of the three hidden states in each phoneme in the sentence.
17. The system of claim 15 , wherein each score is calculated by using a duration for each of the three hidden states in each phoneme in the sentence previously estimated from clean speech.
18. The system of claim 15 , wherein the score is a phoneme-independent evenness score when statistics from clean speech are not available.
19. The system of claim 18 , wherein the phoneme-independent evenness score is given as:
S
=
1
N
∑
n
=
0
N
∑
x
=
{
b
,
m
,
e
}
(
d
n
,
x
-
μ
n
)
2
where d n,x is a duration of an x-th state in an n-th phoneme such that x takes either beginning (b) or middle (m) or ending (e), μ n is an average duration of the three hidden states in the n-th phoneme obtained by d n,x , and N is a total number of phonemes in the sentence.
20. The system of claim 15 , wherein the score is a phoneme-dependent evenness score when statistics from clean speech are available, the phoneme-dependent evenness score given as:
S
=
1
N
∑
n
=
0
N
∑
x
=
{
b
,
m
,
e
}
(
d
n
,
x
-
λ
k
(
n
)
,
x
)
2
where λ k(n),x is an ideal duration for an x-th state of phoneme k(n) to which an n-th phoneme should be referred, estimated from the clean speech in advance and k(n) is a function to map the n-th phoneme to a corresponding phoneme.