Pronunciation conversion apparatus, pitch mark timing extraction apparatus, methods and programs for the same
View Patent ↗Provided is a system which allows a learner who is a non-native speaker of a given language to intuitively improve pronunciation of the language. A pronunciation conversion apparatus includes a conversion section which converts a first feature value corresponding to a first speech signal obtained when a first speaker who speaks a given language as his/her native language speaks another language such that the first feature value approaches a second feature value corresponding to a second speech signal obtained when a second speaker who speaks the other language as his/her native language speaks the other language, each of the first feature value and the second feature value is a feature value capable of representing a difference in pronunciation, and a speech signal obtained from the first feature value after the conversion is presented to the first speaker.
1. A pronunciation conversion apparatus comprising:
an LPC analysis section which determines an LPC residual signal using an LPC coefficient associated with a first speech signal of a speech segment, the first speech signal being obtained when a first speaker who speaks a given language as his/her native language speaks another language;
a pitch mark analysis section which calculates an impulse train corresponding to an EGG glottal closure time;
a cross-correlation estimation section which estimates a delay of the LPC residual signal relative to the EGG glottal closure time;
a pitch mark time estimation section which estimates a pitch mark time based at least on the delay, the impulse train and the LPC residual signal;
an extraction section which extracts a first feature value corresponding to the first speech signal from the pitch mark time and the first speech signal; and
a conversion section which converts the first feature value such that the first feature value approaches a second feature value corresponding to a second speech signal obtained when a second speaker who speaks the other language as his/her native language speaks the other language, wherein
each of the first feature value and the second feature value is a feature value capable of representing a difference in pronunciation, and
the pronunciation conversion apparatus presents a speech signal obtained from the first feature value after the conversion to the first speaker in real time.
2. The pronunciation conversion apparatus according to claim 1 , including:
a speech segment detection section which detects a speech segment of the first speech signal;
the LPC analysis section which obtains the LPC coefficient from the first speech signal of the speech segment through LPC analysis by using the first speech signal and the speech segment;
the pitch mark analysis section which calculates the impulse train corresponding to the EGG glottal closure time by using the first speech signal; and
the cross-correlation estimation section which estimates the delay of the LPC residual signal relative to the EGG glottal closure time by estimating a cross-correlation between the LPC residual signal and the impulse train.
3. The pronunciation conversion apparatus according to claim 1 , wherein
let z denote a root of a prediction polynomial obtained with a linear prediction coefficient, and the conversion section determines a conversion filter
[Formula 16]
[
Formula
16
]
F
(
z
)
=
1
A
′
(
z
)
1
A
(
z
)
from a first vocal tract spectrum A(z) corresponding to the first feature value and a second vocal tract spectrum A′ (z) corresponding to the second feature value, and converts the first feature value by using the conversion filter F(z).
4. The pronunciation conversion apparatus according to claim 3 , wherein
let r p denote a magnitude of an imaginary root of the prediction polynomial obtained with the linear prediction coefficient determined from the first speech signal, let θ p denote an angle of the imaginary root of the prediction polynomial obtained with the linear prediction coefficient, the first vocal tract spectrum A(z) is expressed as
[
Formula
17
]
A
(
z
)
=
∏
p
=
1
P
(
1
-
c
p
z
-
1
)
(
1
-
c
p
*
z
-
1
)
c
p
=
r
p
cos
θ
p
+
j
r
p
sin
θ
p
c
p
*
=
r
p
cos
θ
p
-
j
r
p
sin
θ
p
,
the second vocal tract spectrum A′ (z) is expressed as
[
Formula
18
]
A
′
(
z
)
=
∏
p
=
1
P
(
1
-
c
p
′
z
-
1
)
(
1
-
c
p
′
*
z
-
1
)
,
and
[
Formula
19
]
c
p
′
=
r
p
cos
θ
p
′
+
j
r
p
sin
θ
p
′
c
p
′
*
=
r
p
cos
θ
p
′
-
j
r
p
sin
θ
p
′
or
[
Formula
20
]
c
p
′
=
r
p
′
cos
θ
p
+
jr
p
′
sin
θ
p
c
p
′
*
=
r
p
′
cos
θ
p
-
jr
p
′
sin
θ
p
or
[
Formula
21
]
c
p
′
=
r
p
′
cos
θ
p
′
+
jr
p
′
sin
θ
p
′
c
p
′
*
=
r
p
′
cos
θ
p
′
-
jr
p
′
sin
θ
p
′
is satisfied, let F′ p , denote a formant frequency of the second speech signal, let θ p denote a sampling frequency, let B′ p denote a bandwidth of the formant frequency of the second speech signal, and
[
Formula
22
]
θ
p
′
=
2
π
F
p
′
f
s
[
Formula
23
]
r
p
′
=
e
-
π
B
p
′
f
s
are satisfied.
5. The pronunciation conversion apparatus according to claim 1 , wherein
the conversion section gives a random number to the second feature value to convert the first feature value such that the first feature value approaches the second feature value to which the random number is given.
6. A pronunciation conversion method comprising:
determining an LPC residual signal using an LPC coefficient associated with a first speech signal of a speech segment, the first speech signal being obtained when a first speaker who speaks a given language as his/her native language speaks another language;
calculating an impulse train corresponding to an EGG glottal closure time;
estimating a delay of the LPC residual signal relative to the EGG glottal closure time;
estimating a pitch mark time based at least on the delay, the impulse train and the LPC residual signal;
extracting a first feature value corresponding to the first speech signal from the pitch mark time and the first speech signal; and
a conversion step of causing a conversion section to convert the first feature value such that the first feature value approaches a second feature value corresponding to a second speech signal obtained when a second speaker who speaks the other language as his/her native language speaks the other language, wherein
each of the first feature value and the second feature value is a feature value capable of representing a difference in pronunciation, and
presented, by a pronunciation conversion apparatus, a speech signal obtained from the first feature value after the conversion to the first speaker in real time.
7. A non-transitory computer-readable recording medium that records a program for causing a computer to function as the pronunciation conversion apparatus according to claim 1 .