Voice activity detection based on entropy-energy feature
View Patent ↗This application discloses a voice activity detection method. The method includes receiving speech data, the speech data including a multi-frame speech signal; determining energy and spectral entropy of a frame of speech signal; calculating a square root of the energy of the speech signal and/or calculating a square root of the spectral entropy of the frame of the speech signal; determining a spectral entropy-energy square root of the frame of the speech signal based on at least one of the square root of the energy and the square root of the spectral entropy; and determining that the frame of the speech signal is an unvoiced frame if the spectral entropy-energy square root of the speech signal is less than a first threshold, or that it is a voiced frame if the spectral entropy-energy square root of the speech signal is greater than or equal to the first threshold.
1. A voice activity detection method, executed by a smart terminal and comprising:
receiving speech data, the speech data including a multi-frame speech signal, the multi-frame speech signal including a k th frame speech signal;
determining energy and spectral entropy of the k th frame speech signal;
calculating a square root of the energy of the k th frame speech signal and/or calculating a square root of the spectral entropy of the k th frame speech signal;
determining a spectral entropy-energy square root HE(k) of the k th frame speech signal based on at least one of the square root of the energy and the square root of the spectral entropy,
wherein the spectral entropy-energy square root HE(k) of the k th frame speech signal is determined according to HE(k)=√{square root over (|H(k)−H(0)|*E(k))}, wherein
H
(
0
)
=
1
N
∑
l
=
1
N
H
(
l
)
,
H(k) is a spectral entropy of the k th frame speech signal, E(k) is a short-term energy of the spectral entropy of the k th frame speech signal, k is greater than N, and N represents first N frames in the multi-frame speech signal; and
determining that the k th frame speech signal is an unvoiced frame if the spectral entropy-energy square root of the k th frame speech signal is less than a first threshold, or that the k th frame speech signal is a voiced frame if the spectral entropy-energy square root of the k th frame speech signal is greater than or equal to the first threshold,
wherein the first threshold is adjusted by: multiplying a spectral entropy-energy square root average value HE(0) of first N frames and a special entropy H(0) of the first N frames by a coefficient β if spectral entropy-energy square roots of consecutive P frames speech signal are less than the first threshold, the coefficient β being greater than 0 and less than 1.
2. The method according to claim 1 , wherein the determining a spectral entropy-energy square root of the speech signal based on at least one of the square root of the energy and the square root of the spectral entropy comprises:
multiplying the square root of the energy by the spectral entropy of the speech signal, to obtain the spectral entropy-energy square root of the frame of the speech signal.
3. The method according to claim 1 , wherein the determining a spectral entropy-energy square root of the speech signal based on at least one of the square root of the energy and the square root of the spectral entropy comprises:
multiplying the square root of the spectral entropy by the energy of the speech signal, to obtain the spectral entropy-energy square root of the frame of the speech signal.
4. The method according to claim 1 , wherein the determining a spectral entropy-energy square root of the speech signal based on at least one of the square root of the energy and the square root of the spectral entropy comprises:
multiplying the square root of the energy by the square root of the spectral entropy, to obtain the spectral entropy-energy square root of the frame of the speech signal.
5. The method according to claim 1 ,
wherein the first threshold comprises αHE(0), and a value range of α comprises [1,2].
6. The method according to claim 5 , further comprising:
determining the spectral entropy-energy square root average value according to
HE
(
0
)
=
1
N
∑
l
=
1
N
HE
(
l
)
′
,
HE
(
l
)
′
=
H
(
l
)
*
E
(
l
)
,
wherein H(l) is a spectral entropy of an l th frame of speech signal, and E(l) is short-term energy of the spectral entropy of the l th frame of speech signal.
7. The method according to claim 5 , wherein after the determining a spectral entropy-energy square root average value HE(0) of the first N frames in the multi-frame speech signal, the method further comprises:
determining a variance of the first N frames in the speech signal; and if the variance is greater than a second threshold, performing the operation of determining a spectral entropy-energy square root average value HE(0) of the first N frames in the speech signal again.
8. The method according to claim 1 , wherein after the determining that the frame of the speech signal is a voiced frame if the spectral entropy-energy square root of the speech signal is greater than or equal to the first threshold, the method further comprises:
if consecutive M frames of speech signals are the voiced frames, determining that speech begins, the first frame of speech signal in the M frames of speech signals being the first voiced frame.
9. The method according to claim 1 , wherein after the determining that the frame of the speech signal is an unvoiced frame if the spectral entropy-energy square root of the speech signal is less than a first threshold, the method further comprises:
if consecutive R frames of speech signals are unvoiced frames, determining that a speech ends.
10. The method according to claim 1 , wherein after the determining that the frame of the speech signal is an unvoiced frame if the spectral entropy-energy square root of the speech signal is less than a first threshold, the method further comprises:
if there are no two adjacent frames that are voiced frames in consecutive R frames of speech signals, determining that a speech ends.
11. The method according to claim 1 , wherein after the determining energy and spectral entropy of a frame of speech signal, the method further comprises:
discarding the speech signal if the energy of the speech signal is greater than a second threshold; or
discarding the speech signal if the energy of the speech signal is less than a third threshold.
12. The method according to claim 1 , further comprising:
passing the speech signal through a high pass filter.
13. The method according to claim 1 , further comprising:
performing direct current removal on the k th frame of speech signal.
14. The method according to claim 1 , wherein the multi-frame speech signal includes two adjacent frames, and the two adjacent frames share an overlapping area of acquisition points.
15. A voice activity detection apparatus, comprising: a memory storing computer program instructions; and a processor coupled to the memory and configured to execute the computer program instructions and perform:
receiving speech data, the speech data including a multi-frame speech signal, the multi-frame speech signal including a k th frame speech signal;
determining energy and spectral entropy of the k th frame speech signal;
calculating a square root of the energy the k th frame of the speech signal and/or calculate a square root of the spectral entropy the k th frame speech signal;
determining a spectral entropy-energy square root HE(k) of the k th frame speech signal based on at least one of the square root of the energy and the square root of the spectral entropy,
wherein the spectral entropy-energy square root HE(k) of the k th frame speech signal is determined according to HE(k)=√{square root over (|H(k)−H(0)|*E(k))}, wherein
H
(
0
)
=
1
N
∑
l
=
1
N
H
(
l
)
,
H(k) is a spectral entropy of the k th frame speech signal, E(k) is a short-term energy of the spectral entropy of the k th frame speech signal, k is greater than N, and N represents first N frames in the multi-frame speech signal; and
determining that the k th frame speech signal is an unvoiced frame if the spectral entropy-energy square root of judging unit is less than a first threshold; or that the k th frame speech signal is a voiced frame if the spectral entropy-energy square root of the k th frame speech signal is greater than or equal to the first threshold,
wherein the first threshold is adjusted by: multiplying a spectral entropy-energy square root average value HE(0) of first N frames and a special entropy H(0) of the first N frames by a coefficient β if spectral entropy-energy square roots of consecutive P frames speech signal are less than the first threshold, the coefficient β being greater than 0 and less than 1.
16. The apparatus according to claim 15 , wherein the processor is further configured to execute the computer program instructions and perform:
multiplying the square root of the energy by the square root of the spectral entropy, to obtain the spectral entropy-energy square root of the frame of the speech signal.
17. The apparatus according to claim 15 , wherein the processor is further configured to execute the computer program instructions and perform:
determining a spectral entropy-energy square root average value HE(0) of the first N frames in the multi-frame speech signal, wherein the first threshold comprises αHE(0), a value range of α comprising [1,2].
18. A voice activity detection device, comprising a processor and a memory connected to each other, the memory being configured to store application code, and the processor being configured to call the application code to execute:
receiving speech data, the speech data comprising a multi-frame speech signal, the multi-frame speech signal including a k th frame speech signal;
determining energy and spectral entropy of the k th frame speech signal;
calculating a square root of the energy of the k th frame speech signal and/or calculating a square root of the spectral entropy the k th frame speech signal;
determining a spectral entropy-energy square root HE(k) of the k th frame speech signal based on at least one of the square root of the energy and the square root of the spectral entropy,
wherein the spectral entropy-energy square root HE(k) of the k th frame speech signal is determined according to HE(k)=√{square root over (|H(k)−H(0)|*E(k))}, wherein
H
(
0
)
=
1
N
∑
l
=
1
N
H
(
l
)
,
H(k) is a spectral entropy of the k th frame speech signal, E(k) is a short-term energy of the spectral entropy of the k th frame speech signal, k is greater than N, and N represents first N frames in the multi-frame speech signal; and
determining that the k th frame speech signal is an unvoiced frame if the spectral entropy-energy square root of the k th frame speech signal is less than a first threshold; or determining that the k th frame speech signal is a voiced frame if the spectral entropy-energy square root of the k th frame speech signal is greater than or equal to the first threshold,
wherein the first threshold is adjusted by: multiplying a spectral entropy-energy square root average value HE(0) of first N frames and a special entropy H(0) of the first N frames by a coefficient β if spectral entropy-energy square roots of consecutive P frames speech signal are less than the first threshold, the coefficient β being greater than 0 and less than 1.
19. The device according to claim 18 , wherein the processor is configured to call the application code to execute:
multiplying the square root of the energy by the square root of the spectral entropy, to obtain the spectral entropy-energy square root of the frame of the speech signal.
20. The device according to claim 18 , wherein the processor is configured to call the application code to execute:
determining that a speech ends if there are no two adjacent frames that are voiced frames in consecutive R frames of speech signals.