Method and system for modeling a common-language speech recognition, by a computer, under the influence of a plurality of dialects
View Patent ↗The present invention relates to a method for modeling a common-language speech recognition, by a computer, under the influence of multiple dialects and concerns a technical field of speech recognition by a computer. In this method, a triphone standard common-language model is first generated based on training data of standard common language, and first and second monophone dialectal-accented common-language models are based on development data of dialectal-accented common languages of first kind and second kind, respectively. Then a temporary merged model is obtained in a manner that the first dialectal-accented common-language model is merged into the standard common-language model according to a first confusion matrix obtained by recognizing the development data of first dialectal-accented common language using the standard common-language model. Finally, a recognition model is obtained in a manner that the second dialectal-accented common-language model is merged into the temporary merged model according to a second confusion matrix generated by recognizing the development data of second dialectal-accented common language by the temporary merged model. This method effectively enhances the operating efficiency and admittedly raises the recognition rate for the dialectal-accented common language. The recognition rate for the standard common language is also raised.
1. A computer-implemented method for creating a speech recognition model, the method performed by a processor and comprising the steps of:
generating, by the processor, a standard model of a triphone standard common language based on training data of the standard common language;
generating a first model of a monophone dialectal-accented common language based on development data of a first accent of the standard common language;
creating a temporary language model by merging, through an iterative process, the standard model and the first model with reference to a first confusion matrix formed between the standard model and the development data of the first accent;
generating a second model of a monophone dialectal-accented common language based on development data of a second accent of the standard common language that is different from the first accent; and
creating the speech recognition model by merging, through an iterative process, the second model and the temporary language model with reference to a second confusion matrix formed between the temporary model and the development data of the second accent.
2. A modeling method according to claim 1 , wherein a probability density function used in the merging of the temporary language model is expressed by
p ′( x|s )=λ 1 p ( x|s )+(1−λ 1 ) p ( x|d 1 ) p ( d 1 |s )
where x is an observation feature vector of voice to be recognized, s is a hidden Markov state in the standard model, d 1 is a hidden Markov state in the first model, and λ 1 is a linear interpolating coefficient such that 0<λ 1 <1, and
wherein a probability density function used in the merging of the speech recognition model is expressed by
p
′′
(
x
|
s
)
=
∑
k
=
1
K
w
k
(
sc
)
′
N
k
(
sc
)
(
·
)
+
∑
m
=
1
M
∑
n
=
1
N
w
mn
(
dc
1
)
′
N
mn
(
dc
1
)
(
·
)
+
∑
p
=
1
P
∑
q
=
1
Q
w
pq
(
dc
2
)
′
N
pq
(
dc
2
)
(
·
)
where w k (sc)′ is a mixture weight for the hidden Markov state of the standard model, w mn (dc1)′ is mixture weight for the hidden Markov state of the first model, w pq (dc2)′ is a mixture weight for the hidden Markov state of the second model, K is the number of Gaussian mixtures for Hidden Markov Model state s in the standard model, N k (sc) (•) is an element of Gaussian mixture for Hidden Markov Model state s, M is the number of d 1 that is considered as the pronunciation variants occurring between the first model for d 1 and the standard model, N is the number of Gaussian mixtures for Hidden Markov Model state d 1 in the first model, N mn (dc1) (•) is an element of Gaussian mixture for Hidden Markov Model state d 1 , P is the number of d 2 that is considered as the pronunciation variants occurring between the second model for d 2 and the standard model, Q is the number of Gaussian mixtures for Hidden Markov Model state d 2 in the second model, N pq (dc2) (•) is an element of Gaussian mixture for Hidden Markov Model state d 2 .
3. A non-transitory, computer-readable medium encoded with a program, executable by a computer, for creating a speech recognition model, the program comprising functions executed by a processor of:
generating a standard model of a triphone standard common language based on training data of the standard common language;
generating a first model of a monophone dialectal-accented common language based on development data of a first accent of the standard common language;
creating a temporary language model by merging, through an iterative process, the standard model and the first model with reference to a first confusion matrix formed between the standard model and the development data of the first accent;
generating a second model of a monophone dialectal-accented common language based on development data of a second accent of the standard common language that is different from the first accent; and
creating the speech recognition model by merging, through an iterative process, the second model and the temporary language model with reference to a second confusion matrix formed between the temporary model and the development data of the second accent.
4. A non-transitory, computer-readable medium according to claim 3 , wherein a probability density function used in the merging of the temporary language model is expressed by
p ′( x|s )=λ 1 p ( x|s )+(1−λ 1 ) p ( x|d 1 ) p ( d 1 |s )
where x is an observation feature vector of voice to be recognized, s is a hidden Markov state in the standard model, d 1 is a hidden Markov state in the first model, and λ 1 is a linear interpolating coefficient such that 0<λ 1 <1, and
wherein a probability density function used in the merging of the speech recognition model is expressed by
p
′′
(
x
|
s
)
=
∑
k
=
1
K
w
k
(
sc
)
′
N
k
(
sc
)
(
·
)
+
∑
m
=
1
M
∑
n
=
1
N
w
mn
(
dc
1
)
′
N
mn
(
dc
1
)
(
·
)
+
∑
p
=
1
P
∑
q
=
1
Q
w
pq
(
dc
2
)
′
N
pq
(
dc
2
)
(
·
)
where w k (sc)′ is a mixture weight for the hidden Markov state of the standard model, w mn (dc1)′ is mixture weight for the hidden Markov state of the first model, w pq (dc2)′ is a mixture weight for the hidden Markov state of the second model, K is the number of Gaussian mixtures for Hidden Markov Model state s in the standard model, N k (sc) (•) is an element of Gaussian mixture for Hidden Markov Model state s, M is the number of d 1 that is considered as the pronunciation variants occurring between the first model for d 1 and the standard model, N is the number of Gaussian mixtures for Hidden Markov Model state d 1 in the first model, N mn (dc1) (•) is an element of Gaussian mixture for Hidden Markov Model state d 1 , P is the number of d 2 that is considered as the pronunciation variants occurring between the second model for d 2 and the standard model, Q is the number of Gaussian mixtures for Hidden Markov Model state d 2 in the second model, N pq (dc2) (•) is an element of Gaussian mixture for Hidden Markov Model state d 2 .
5. A computer-implemented method for creating a speech recognition model, the method performed by a processor and comprising:
generating, by the processor, a standard model of a triphone standard common language based on training data of the standard common language;
generating a first model of a monophone dialectal-accented common language based on development data of a first accent of the standard common language;
creating a temporary language model by merging, through an iterative process, the standard model and the first model with reference to a first confusion matrix formed between the standard model and the development data of the first accent;
generating a second model to nth model, where n is a natural number greater than or equal to 2, of a monophone dialectal-accented common language based on development data of a second accent to nth accent, respectively, of the standard common language that are different from each other; and
creating the speech recognition model by merging, through an iterative process, an ith model, where i is an integer such that 1<i≦n, and an (i−1)th temporary language model with reference to an ith confusion matrix formed between the (i−1)th temporary model and the development data of the ith accent.
6. A non-transitory, computer-readable medium encoded with a program, executable by a computer, for creating a speech recognition model, the program comprising functions performed by a processor of:
generating a standard model of a triphone standard common language based on training data of the standard common language;
generating a first model of a monophone dialectal-accented common language based on development data of a first accent of the standard common language;
creating a temporary language model by merging, through an iterative process, the standard model and the first model with reference to a first confusion matrix formed between the standard model and the development data of the first accent;
generating a second model to nth model, where n is a natural number greater than or equal to 2, of a monophone dialectal-accented common language based on development data of a second accent to nth accent, respectively, of the standard common language that are different from each other; and
creating the speech recognition model by merging, through an iterative process, an ith model, where i is an integer such that 1<i≦n, and an (i−1)th temporary language model with reference to an ith confusion matrix formed between the (i−1)th temporary model and the development data of the ith accent.
7. A model generating unit, controlled by a control unit, system for creating a speech recognition model, the model generating unit comprising:
a standard common-language training database, which stores training data of a triphone standard common language;
a first development database, which stores first development data of a monophone dialectal-accented common language based on a first accent of the standard common language;
a second development database, which stores second development data of a monophone dialectal-accented common language based on a second accent of the standard common language that is different from the first accent;
a standard model generator, which generates a standard model based on training data stored in the standard common-language training database;
a first model generator, which generates a first model based on development data of a first accent of the standard common language stored in the first development database;
a temporary model merging unit, which creates a temporary language model by merging, through an iterative process, the standard model and the first model with reference to a first confusion matrix formed, by a first confusion matrix generator, between the standard model and the development data of the first accent stored in the first development database;
a second model generator, which generates a second model based on development data of a second accent of the standard common language stored in the second development database; and
a speech recognition model merging unit, which creates the speech recognition model by merging, through an iterative process, the second model and the temporary language model with reference to a second confusion matrix formed, by a second confusion matrix generator, between the temporary model and the development data of the second accent stored in the second development database.
8. The modeling generating unit according to claim 7 , wherein a probability density function used by the temporary model merging unit for merging of the temporary language model is expressed by
p ′( x|s )=λ 1 p ( x|s )+(1−λ 1 ) p ( x|d 1 ) p ( d 1 |s )
where x is an observation feature vector of voice to be recognized, s is a hidden Markov state in the standard model, d 1 is a hidden Markov state in the first model, and λ 1 is a linear interpolating coefficient such that 0<λ 1 <1, and
wherein a probability density function used by the speech recognition model merging unit for merging of the speech recognition model is expressed by
p
′′
(
x
|
s
)
=
∑
k
=
1
K
w
k
(
sc
)
′
N
k
(
sc
)
(
·
)
+
∑
m
=
1
M
∑
n
=
1
N
w
mn
(
dc
1
)
′
N
mn
(
dc
1
)
(
·
)
+
∑
p
=
1
P
∑
q
=
1
Q
w
pq
(
dc
2
)
′
N
pq
(
dc
2
)
(
·
)
where w k (sc)′ is a mixture weight for the hidden Markov state of the standard model, w mn (dc1)′ is mixture weight for the hidden Markov state of the first model, w pq (dc2)′ is a mixture weight for the hidden Markov state of the second model, K is the number of Gaussian mixtures for Hidden Markov Model state s in the standard model, N k (sc) (•) is an element of Gaussian mixture for Hidden Markov Model state s, M is the number of d 1 that is considered as the pronunciation variants occurring between the first model for d 1 and the standard model, N is the number of Gaussian mixtures for Hidden Markov Model state d 1 in the first model, N mn (dc1) (•) is an element of Gaussian mixture for Hidden Markov Model state d 1 , P is the number of d 2 that is considered as the pronunciation variants occurring between the second model for d 2 and the standard model, Q is the number of Gaussian mixtures for Hidden Markov Model state d 2 in the second model, N pq (dc2) (•) is an element of Gaussian mixture for Hidden Markov Model state d 2 .