Voice enhancement method, apparatus and system, and computer-readable storage medium
Disclosed are a voice enhancement method, apparatus and system and a computer-readable storage medium. The method includes acquiring a time-domain microphone signal and a time-domain bone conduction signal at the current moment; determining whether the signals are voice signals, if yes, performing a noise cancellation processing to the time-domain microphone signal by a pre-established DNN noise cancellation model, performing a frequency-domain noise cancellation processing to the time-domain bone conduction signal, if not, setting an output signal at the current moment as zero; performing a high-pass filtering processing to the time-domain microphone signal from which noise has been cancelled, to obtain a first output time-domain signal, performing a low-pass filtering processing to the time-domain bone conduction signal from which noise has been cancelled, to obtain a second output time-domain signal; obtaining an output time-domain signal at the current moment according to the first and second output time-domain signals.
1 . A voice enhancement method, comprising:
acquiring a time-domain microphone signal and a time-domain bone conduction signal at the current moment;
determining whether the time-domain microphone signal and the time-domain bone conduction signal are voice signals, if the time-domain microphone signal and the time-domain bone conduction signal are voice signals, performing a noise cancellation processing to the time-domain microphone signal by a pre-established DNN noise cancellation model so as to obtain a time-domain microphone signal from which noise has been cancelled, and performing a frequency-domain noise cancellation processing to the time-domain bone conduction signal so as to obtain a time-domain bone conduction signal from which noise has been cancelled, if the time-domain microphone signal and the time-domain bone conduction signal are not voice signals, setting an output signal at the current moment as zero;
performing a high-pass filtering processing to the time-domain microphone signal from which noise has been cancelled, so as to obtain a first output time-domain signal, and performing a low-pass filtering processing to the time-domain bone conduction signal from which noise has been cancelled, so as to obtain a second output time-domain signal; and
obtaining an output time-domain signal at the current moment according to the first output time-domain signal and the second output time-domain signal.
2 . The voice enhancement method of claim 1 , wherein performing a frequency-domain noise cancellation processing to the time-domain bone conduction signal, so as to obtain a time-domain bone conduction signal from which noise has been cancelled comprises:
converting the time-domain bone conduction signal into a frequency-domain bone conduction signal through time-to-frequency transformation;
performing a frequency-domain noise cancellation processing to the frequency-domain bone conduction signal so as to obtain a frequency-domain bone conduction signal from which noise has been cancelled; and
determining whether a bandwidth of the frequency-domain bone conduction signal from which noise has been cancelled reaches a preset bandwidth, if the bandwidth of the frequency-domain bone conduction signal from which noise has been cancelled reaches the preset bandwidth, directly performing frequency-to-time inverse transformation to the frequency-domain bone conduction signal from which noise has been cancelled so as to obtain the time-domain bone conduction signal from which noise has been cancelled, if the bandwidth of the frequency-domain bone conduction signal from which noise has been cancelled does not reach the preset bandwidth, expanding the bandwidth of the frequency-domain bone conduction signal from which noise has been cancelled by using a pre-established DNN bandwidth expanding model so that the expanded bandwidth reaches the preset bandwidth, and performing frequency-to-time transformation to the expanded frequency-domain bone conduction signal so as to obtain the time-domain bone conduction signal from which noise has been cancelled.
3 . The voice enhancement method of claim 1 , wherein performing a noise cancellation processing to the time-domain microphone signal by a pre-established DNN noise cancellation model so as to obtain a time-domain microphone signal from which noise has been cancelled comprises:
performing a time-to-frequency transformation to the time-domain microphone signal to obtain a corresponding frequency-domain microphone signal;
extracting a first signal feature of the frequency-domain microphone signal, and processing the first signal feature by using the pre-established DNN noise cancellation model, so as to obtain first gains corresponding to first frequency points of the frequency-domain microphone signal respectively;
calculating the product of spectral signals corresponding to the first frequency points in the frequency-domain microphone signal and corresponding first gains, to obtain spectral signals from which noise has been cancelled corresponding to the first frequency points respectively, so as to obtain a frequency-domain microphone signal from which noise has been cancelled; and
performing a frequency-to-time transformation to the frequency-domain microphone signal from which noise has been cancelled to obtain the time-domain microphone signal from which noise has been cancelled.
4 . The voice enhancement method of claim 1 , wherein determining whether the time-domain microphone signal and the time-domain bone conduction signal are voice signals comprises:
performing a voice activation detection to the time-domain bone conduction signal to determine whether the time-domain bone conduction signal is a voice signal; and
when the time-domain bone conduction signal is a voice signal, the time-domain microphone signal is a voice signal.
5 . The voice enhancement method of claim 4 , wherein performing a voice activation detection to the time-domain bone conduction signal to determine whether the time-domain bone conduction signal is a voice signal comprises:
calculating a zero-crossing rate and a pitch period corresponding to the time-domain bone conduction signal;
performing time-to-frequency transformation to the time-domain bone conduction signal to obtain a frequency-domain bone conduction signal;
calculating a spectral energy and a spectral centroid corresponding to the frequency-domain bone conduction signal;
comprehensively determining the zero-crossing rate, the pitch period, the spectral energy and the spectral centroid to obtain a voice activation detection flag bit corresponding to the time-domain bone conduction signal; and
determining whether the time-domain bone conduction signal is a voice signal according to the voice activation detection flag bit.
6 . The voice enhancement method of claim 5 , wherein comprehensively determining the zero-crossing rate, the pitch period, the spectral energy and the spectral centroid to obtain a voice activation detection flag bit corresponding to the time-domain bone conduction signal comprises:
determining whether the spectrum energy is less than a first preset value, if the spectrum energy is less than the first preset value, the voice activation detection flag bit corresponding to the time-domain bone conduction signal is 0, if the spectrum energy is not less than the first preset value, proceed to a next step for determination;
determining whether the zero-crossing rate is greater than a second preset value, if the zero-crossing rate is greater than the second preset value, the voice activation detection flag bit corresponding to the time-domain bone conduction signal is 0, if the zero-crossing rate is not greater than the second preset value, proceed to a next step for determination;
determining whether the pitch period is greater than a third preset value or less than a fourth preset value, if the pitch period is greater than the third preset value or less than the fourth preset value, the voice activation detection flag bit corresponding to the time-domain bone conduction signal is 0, if the pitch period is not greater than the third preset value and not less than the fourth preset value, proceed to a next step for determination;
determining whether the spectral centroid is greater than a fifth preset value, if the spectral centroid is greater than the fifth preset value, the voice activation detection flag bit corresponding to the time-domain bone conduction signal is 0, if the spectral centroid is not greater than the fifth preset value, the voice activation detection flag bit corresponding to the time-domain bone conduction signal is 1; and
determining whether the time-domain bone conduction signal is a voice signal according to the voice activation detection flag bit comprises:
when the voice activation detection flag bit is 1, the time-domain bone conduction signal is a voice signal; and
when the voice activation detection flag bit is 0, the current time-domain bone conduction signal is a noise signal.
7 . The voice enhancement method of claim 1 , wherein obtaining an output time-domain signal at the current moment according to the first output time-domain signal and the second output time-domain signal comprises:
combining the first output time-domain signal and the second output time-domain signal according to a first weight coefficient and a second weight coefficient to obtain a combined time-domain signal; and
dynamically adjusting the combined time-domain signal so that the adjusted time-domain signal is within a preset range, and taking the adjusted time-domain signal as the output time-domain signal corresponding to the current time.
8 . A voice enhancement apparatus, comprising:
an acquisition module for acquiring a time-domain microphone signal and a time-domain bone conduction signal at the current moment;
a determination module for determining whether the time-domain microphone signal and the time-domain bone conduction signal are voice signals, if the time-domain microphone signal and the time-domain bone conduction signal are voice signals, activate a noise reduction module, if the time-domain microphone signal and the time-domain bone conduction signal are not voice signals, activate a zeroing module;
the noise reduction module for performing a noise cancellation processing to the time-domain microphone signal by a pre-established DNN noise cancellation model so as to obtain a time-domain microphone signal from which noise has been cancelled, and performing a frequency-domain noise cancellation processing to the time-domain bone conduction signal so as to obtain a time-domain bone conduction signal from which noise has been cancelled;
the zeroing module for setting an output signal at the current moment as zero;
a filtering module for performing a high-pass filtering processing to the time-domain microphone signal from which noise has been cancelled, so as to obtain a first output time-domain signal, and performing a low-pass filtering processing to the time-domain bone conduction signal from which noise has been cancelled, so as to obtain a second output time-domain signal; and
a combining module for obtaining an output time-domain signal at the current moment according to the first output time-domain signal and the second output time-domain signal.
9 . A voice enhancement system, comprising:
a memory for storing a computer program; and
a processor for implementing steps of the voice enhancement method of claim 1 when executing the computer program.
10 . A Non-transitory computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, steps of the voice enhancement method of claim 1 are implemented.