Sound signal refinement method, sound signal decode method, apparatus thereof, program, and storage medium
There is provided a technology that improves, in a case where there is a sound signal obtained from a different code that is different from a code from which a decoded sound signal is obtained and that is derived from the same sound signal, the decoded sound signal by using the sound signal obtained from the different code. A signal (hereinafter, referred to as a decoded sound common signal) obtained by downmixing a decoded sound signal of each channel is subjected to signal purification using a monaural decoded sound signal to thereby generate a purified common signal, and in each channel, the decoded sound common signal is subtracted from the decoded sound signal and the purified common signal is added thereto, to thereby generate a purified decoded sound signal.
1 . A sound signal purification method for obtaining a purified decoded sound signal representing a sound signal of a channel of stereo,
the sound signal purification method comprising:
a decoded sound common signal estimation step of obtaining, for each frame, a decoded sound common signal {circumflex over ( )}Y M that is a signal common to all channels of the stereo by using at least all of one or more and N or less n-th channel decoded sound signals {circumflex over ( )}X n ,
wherein n represents each integer of 1 or more and N or less, the n-th channel decoded sound signal {circumflex over ( )}X n represents a decoded sound signal of the each channel of the stereo, the n-th channel decoded sound signal {circumflex over ( )}X n is obtained by decoding a stereo code CS and a monaural decoded sound signal {circumflex over ( )}X M represents a monaural decoded sound signal, the monaural decoded sound signal {circumflex over ( )}X M is obtained by decoding a monaural code CM that is a code different from the stereo code CS, and the n-th channel decoded sound signal {circumflex over ( )}X n is obtained by decoding the stereo code CS without using either information obtained by decoding the monaural code CM or the monaural code CM;
a common signal purification step of obtaining, for the each frame and for each corresponding sample t, a sequence based on a value ˜ y M (t)=(1−α M )×{circumflex over ( )}y M (t)+α M ×{circumflex over ( )}x M (t) as a purified common signal ˜ Y M ,
wherein the value ˜ y M (t) is obtained by adding a value α M ×{circumflex over ( )}x M (t) and a value (1−α M )×{circumflex over ( )}y M (t), the value α M ×{circumflex over ( )}x M (t) is obtained by multiplying a common signal purification weight α m by a sample value {circumflex over ( )}x M (t) of the monaural decoded sound signal {circumflex over ( )}x M , the value (1−α M )×{circumflex over ( )}y M (t) is obtained by multiplying a value (1−α M ) by a sample value {circumflex over ( )}y M (t) of the decoded sound common signal {circumflex over ( )}Y M , and the value (1−α M ) is obtained by subtracting the common signal purification weight α M from 1;
an n-th channel separation combination weight estimation step of obtaining, for the each frame with respect to the each channel n, a normalized inner product value for the decoded sound common signal {circumflex over ( )}Y M of the n-th channel decoded sound signal {circumflex over ( )}X n as an n-th channel separation combination weight β n ; and
an n-th channel separation combination step of obtaining, for the each frame and for each corresponding sample t with respect to the each channel n, a sequence based on a value ˜ x n (t)={circumflex over ( )}x n (t)−β n ×{circumflex over ( )}y M (t)+β n ×{circumflex over ( )}y M (t) as an n-th channel purified decode sound signal ˜ X n ,
wherein the value ˜ x n (t) is obtained by subtracting a value β n ×{circumflex over ( )}y M (t) from a sample value {circumflex over ( )}X n (t) of the n-th channel decoded sound signal {circumflex over ( )}X n and adding a value β n ×{circumflex over ( )}y M (t), the value β n ×{circumflex over ( )}y M (t) is obtained by multiplying the n-th channel separation combination weight β n by the sample value {circumflex over ( )}y M (t) of the decoded sound common signal {circumflex over ( )}Y M , and the value β n × ˜ y M (t) is obtained by multiplying the n-th channel separation combination weight β n by a sample value ˜ y M (t) of the purified common signal ˜ Y M .
2 . The sound signal purification method according to claim 1 , wherein
the decoded sound common signal estimation step further comprises operations to
obtain a weighting coefficient that minimizes a difference between a weighted average of the decoded sound signals {circumflex over ( )}X 1 , . . . , {circumflex over ( )}X N of all channels from a first to an N-th channel and the monaural decoded sound signal {circumflex over ( )}X M , and
obtain, as the decoded sound common signal {circumflex over ( )}Y M , the weighted average of the decoded sound signals {circumflex over ( )}X 1 , . . . , {circumflex over ( )}X N of all the channels from the first to the N-th channel using the obtained weighting coefficient.
3 . The sound signal purification method according to claim 1 , wherein
a number N of the channels is two, and
the decoded sound common signal estimation step further comprises operations to
use a sample value {circumflex over ( )}x 1 (t) of a first channel decoded sound signal {circumflex over ( )}X 1 , a sample value {circumflex over ( )}x 2 (t) of a second channel decoded sound signal {circumflex over ( )}X 2 , and an inter-channel correlation coefficient γ of a first channel and a second channel to obtain,
on a basis of information indicating which of the first channel and the second channel is preceding,
a sequence based on {circumflex over ( )}y M (t)=((1+γ)/2)×{circumflex over ( )}x 1 (t)+((1−γ)/2)×{circumflex over ( )}x 2 (t) as the decoded sound common signal {circumflex over ( )}Y M in a case where the first channel is preceding,
a sequence based on {circumflex over ( )}y M (t)=((1−γ)/2)×{circumflex over ( )}x 1 (t)+((1+γ)/2)×{circumflex over ( )}x 2 (t) as the decoded sound common signal {circumflex over ( )}Y M in a case where the second channel is preceding, or
a sequence based on {circumflex over ( )}y M (t)={circumflex over ( )}x 1 (t)+{circumflex over ( )}x 2 (t))/2 as the decoded sound common signal {circumflex over ( )}Y M in a case where none of the channels is preceding.
4 . The sound signal purification method according to claim 1 , further comprising
a common signal purification weight estimation step of obtaining,
for the each frame,
the common signal purification weight α M by
α
M
=
2
-
2
b
m
T
2
-
2
b
m
T
+
2
-
2
b
M
T
using a number of samples T per frame, a number of bits b m corresponding to a common signal in a number of bits of the stereo code CS, and a number of bits b M of the monaural code CM.
5 . The sound signal purification method according to claim 1 , further comprising
a common signal purification weight estimation step of obtaining,
for the each frame,
a value that is larger than 0 and smaller than 1, 0.5 when b m and b m are equal, closer to 0 than 0.5 as b m is larger than b M , and closer to 1 than 0.5 as b m is larger than b m by using at least a number of bits b m corresponding to a common signal in a number of bits of the stereo code CS, and a number of bits b M of the monaural code CM, as the common signal purification weight α M .
6 . The sound signal purification method according to claim 1 , further comprising
a common signal purification weight estimation step of obtaining,
for the each frame,
a value c M ×r M obtained by multiplying
a normalized inner product value r M for the monaural decoded sound signal {circumflex over ( )}X M of the decoded sound common signal {circumflex over ( )}Y M by
a correction coefficient c M obtained by
c
M
=
2
-
2
b
m
T
2
-
2
b
m
T
+
2
-
2
b
M
T
using a number of samples T per frame, a number of bits b m corresponding to a common signal in a number of bits of the stereo code CS, and a number of bits b m of the monaural code CM, as the common signal purification weight α M .
7 . The sound signal purification method according to claim 1 , further comprising
a common signal purification weight estimation step of obtaining,
for the each frame,
with a number of bits corresponding to a common signal in a number of bits of the stereo code CS as b m and a number of bits of the monaural code CM as b M ,
a value c M ×r M obtained by multiplying
r M that is a value closer to 1 as a correlation between the decoded sound common signal {circumflex over ( )}Y M and the monaural decoded sound signal {circumflex over ( )}X M is higher, and closer to 0 as the correlation is lower by
a correction coefficient c M that is a value larger than 0 and smaller than 1, 0.5 when b m and b M are equal, closer to 0 than 0.5 as b m is larger than b M , and closer to 1 than 0.5 as b m is smaller than b M , as the common signal purification weight α M .
8 . The sound signal purification method according to claim 1 , wherein
T is a number of samples per frame and each of ε m and ε M is a value larger than 0 and smaller than 1, and
the sound signal purification method further comprises
a common signal purification weight estimation step of obtaining,
for the each frame,
a value c M ×r M obtained by multiplying
a normalized inner product value r M obtained by
r M =E m (0)/ E M (0)
using an inner product value E m (0) obtained by
E
m
(
0
)
=
ϵ
m
E
m
(
-
1
)
+
(
1
-
ϵ
m
)
T
∑
t
=
1
T
y
ˆ
M
(
t
)
x
ˆ
M
(
t
)
using each sample value {circumflex over ( )}y M (t) of the decoded sound common signal {circumflex over ( )}Y M , each sample value {circumflex over ( )}x M (t) of the monaural decoded sound signal {circumflex over ( )}X M , and an inner product value E m (−1) of a previous frame, and
energy E M (0) of the monaural decoded sound signal obtained by
E
M
(
0
)
=
ϵ
M
E
M
(
-
1
)
+
(
1
-
ϵ
M
)
T
∑
t
=
1
T
x
ˆ
M
(
t
)
x
ˆ
M
(
t
)
using the each sample value {circumflex over ( )}X M (t) of the monaural decoded sound signal {circumflex over ( )}X M and energy E M (−1) of the monaural decoded sound signal of the previous frame, by
a correction coefficient c M obtained by
c
M
=
2
-
2
b
m
T
2
-
2
b
m
T
+
2
-
2
b
M
T
using a number of samples T per frame, a number of bits b m corresponding to a common signal in a number of bits of the stereo code CS, and a number of bits b M of the monaural code CM, as the common signal purification weight α M .
9 . The sound signal purification method according to claim 6 , wherein
the common signal purification weight estimation step further comprises operations to
obtain a value λ×c M ×r M obtained by multiplying the normalized inner product value r M , the correction coefficient c M , and λ that is a predetermined value larger than 0 and smaller than 1 as the common signal purification weight α M .
10 . The sound signal purification method according to claim 6 , wherein
a number N of the channels is two, and
the common signal purification weight estimation step further comprises operations to
obtain a value γ×c M ×r M obtained by multiplying the normalized inner product value r M , the correction coefficient c M , and an inter-channel correlation coefficient γ that is a correlation coefficient between a first channel decoded sound signal and a second channel decoded sound signal as the common signal purification weight α M .
11 . A sound signal decoding method comprising the sound signal purification method according to claim 1 as a sound signal purification step, the sound signal decoding method further comprising:
a stereo decoding step of decoding the stereo code CS to obtain the n-th channel decoded sound signal {circumflex over ( )}X n of the each channel n without using either information obtained by decoding the monaural code CM or the monaural code CM; and
a monaural decoding step of decoding the monaural code CM to obtain the monaural decoded sound signal {circumflex over ( )}X M .
12 . A sound signal purification device for obtaining a purified decoded sound signal representing a sound signal of a channel of stereo,
the sound signal purification device further comprises
a decoded sound common signal estimation circuitry configured to obtain, for each frame, a decoded sound common signal {circumflex over ( )}Y M that is a signal common to all channels of the stereo by using at least all of one or more and N or less n-th channel decoded sound signals {circumflex over ( )}X n ,
wherein n represents each integer of 1 or more and N or less, the n-th channel decoded sound signal {circumflex over ( )}X n represents a decoded sound signal of the each channel of the stereo, the n-th channel decoded sound signal {circumflex over ( )}X n is obtained by decoding a stereo code CS and a monaural decoded sound signal {circumflex over ( )}X M represents a monaural decoded sound signal, the monaural decoded sound signal {circumflex over ( )}X M is obtained by decoding a monaural code CM that is a code different from the stereo code CS, and the n-th channel decoded sound signal {circumflex over ( )}X n is obtained by decoding the stereo code CS without using either information obtained by decoding the monaural code CM or the monaural code CM;
a common signal purification circuitry configured to obtain, for the each frame and for each corresponding sample t, a sequence based on a value ˜ y M (t)=(1−α M )×{circumflex over ( )}y M (t)+α M ×{circumflex over ( )}X M (t) as a purified common signal ˜ Y M , wherein the value ˜ y M (t) is obtained by adding a value α M ×{circumflex over ( )}X M (t) and a value (1−α M )×{circumflex over ( )}y M (t), the value α M ×{circumflex over ( )}X M (t) is obtained by multiplying a common signal purification weight am by a sample value {circumflex over ( )}X M (t) of the monaural decoded sound signal {circumflex over ( )}X M , the value (1−α M )×{circumflex over ( )}y M (t) is obtained by multiplying a value (1−α M ) by a sample value {circumflex over ( )}y M (t) of the decoded sound common signal {circumflex over ( )}Y M , and the value (1−α M ) is obtained by subtracting the common signal purification weight α M from 1;
an n-th channel separation combination weight estimation circuitry configured to obtain, for the each frame with respect to the each channel n, a normalized inner product value for the decoded sound common signal {circumflex over ( )}Y M of the n-th channel decoded sound signal {circumflex over ( )}X n as an n-th channel separation combination weight β n ; and
an n-th channel separation combination circuitry configured to obtain, for the each frame and for each corresponding sample t with respect to the each channel n, a sequence based on a value ˜ x n (t)={circumflex over ( )}X n (t)−β n ×{circumflex over ( )}y M (t)+β n × ˜ y M (t) as an n-th channel purified decoded sound signal ˜ X n , wherein the value ˜ x n (t) is obtained by subtracting a value β n ×{circumflex over ( )}y M (t) from a sample value {circumflex over ( )}x n (t) of the n-th channel decoded sound signal {circumflex over ( )}X n and adding a value β n × ˜ y M (t), the value β n ×{circumflex over ( )}y M (t) is obtained by multiplying the n-th channel separation combination weight β n by the sample value {circumflex over ( )}y M (t) of the decoded sound common signal {circumflex over ( )}Y M , and the value β n × ˜ y M (t) is obtained by multiplying the n-th channel separation combination weight β n by a sample value ˜ y M (t) of the purified common signal ˜ Y M .
13 . A sound signal decoding device comprising the sound signal purification device according to claim 12 as a sound signal purification circuitry, the sound signal decoding device further comprising:
a stereo decoding circuitry configured to decode the stereo code CS to obtain the n-th channel decoded sound signal {circumflex over ( )}X n of the each channel n without using either information obtained by decoding the monaural code CM or the monaural code CM; and
a monaural decoding circuitry configured to decode the monaural code CM to obtain the monaural decoded sound signal {circumflex over ( )}X M .
14 . A non-transitory recording medium recording a program for causing a computer to execute the sound signal purification method according to claim 1 .