Granted Patent B2

US 12,236,974 · App. 17/872,909 · Granted Feb 25, 2025

Inventors: Libiao Yu (Beijing, CN); Guochang Zhang (Beijing, CN); Jianqiang Wei (Beijing, CN)

Assignee: Beijing Baidu Netcom Science Technology Co., Ltd.

G10L25/51G10L15/16G10L21/0224G10L25/18G10L2015/0631G10L2021/02082

Abstract

Claims (61)

1. A method for processing a signal, the method comprising:

acquiring a reference signal of a to-be-tested voice, the reference signal being a signal output to a voice output device, wherein the voice output device outputs the to-be-tested voice after obtaining the reference signal;

receiving, from a voice input device, an echo signal of the to-be-tested voice, the echo signal being a signal of the to-be-tested voice collected by the voice input device;

performing signal preprocessing on the reference signal and the echo signal respectively; and

inputting the processed reference signal and the processed echo signal into a pre-trained time delay estimation model, to obtain a time difference between the reference signal and the echo signal output by the time delay estimation model, the time delay estimation model being used to represent a corresponding relationship between the reference signal, the echo signal and the time difference, wherein the time delay estimation model is configured to extract a feature of the reference signal and a feature of the echo signal, and is obtained by training operations based on long-term correlations between features of reference signals and features of echo signals; the time delay estimation model comprises: a convolutional neural network, a temporal convolutional network, and a fully connected layer, wherein the convolutional neural network, the temporal convolutional network and the fully connected layer are connected in sequence, the convolutional neural network is configured to extract and deeply fuse the feature of the reference signal and the feature of the echo signal, the temporal convolutional network is configured to learn the long-term correlation between the feature of the reference signal and the feature of the echo signal, and the fully connected layer is configured to extract the time delay between the reference signal and the echo signal.

2. The method according to claim 1 , wherein, before acquiring the reference signal of the to-be-tested voice, the method further comprises:

acquiring a plurality of constant-amplitude sweep-frequency signals;

inputting at least one sweep-frequency signal into the voice output device, and acquiring an output signal of the sweep-frequency signal in the voice input device;

detecting whether the voice output device and the voice input device are non-linear devices, based on the sweep-frequency signal and the output signal corresponding to the sweep-frequency signal; and

in response to determining that the voice output device and the voice input device are non-linear devices, acquiring the reference signal of the to-be-tested voice.

3. The method according to claim 2 , wherein the method further comprises:

in response to determining that the voice output device and the voice input device are not non-linear devices, calculating the time difference between the reference signal and the echo signal using a time delay estimation algorithm.

4. The method according to claim 1 , wherein the time delay estimation model is obtained through training operations as follows:

acquiring training samples, wherein each training sample comprises a sample signal and an echo signal corresponding to the sample signal; and

performing the training operations: inputting a sample signal and an echo signal corresponding to the sample signal into a first convolutional neural network, to obtain a feature vector of the sample signal and a feature vector of the echo signal corresponding to the sample signal; inputting the feature vector of the sample signal and the feature vector of the echo signal corresponding to the sample signal into a second convolutional neural network, to obtain correlation information of the sample signal and the echo signal corresponding to the sample signal within a preset time period; inputting the correlation information of the sample signal and the echo signal corresponding to the sample signal within the preset time period into a fully connected layer, to obtain a time difference between the sample signal and the echo signal corresponding to the sample signal; in response to determining that the first convolutional neural network, the second convolutional neural network, and the fully connected layer meet a training completion condition, using the first convolutional neural network, the second convolutional neural network, and the fully connected layer as the time delay estimation model.

5. The method according to claim 1 , wherein the performing signal preprocessing on the reference signal and the echo signal respectively, comprises:

performing subband decomposition and catenation on the reference signal and the echo signal respectively; and

obtaining magnitude spectrograms of the catenated signals to obtain the processed reference signal and the processed echo signal.

6. The method according to claim 1 , wherein the performing signal preprocessing on the reference signal and the echo signal respectively, comprises:

performing frequency domain transformation on the reference signal and the echo signal respectively, to obtain the processed reference signal and the processed echo signal.

7. An apparatus for processing a signal, the apparatus comprising:

at least one processor; and

a memory communicatively connected to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:

acquiring a reference signal of a to-be-tested voice, the reference signal being a signal output to a voice output device, wherein the voice output device outputs the to-be-tested voice after obtaining the reference signal;

receiving, from a voice input device, an echo signal of the to-be-tested voice, the echo signal being a signal of the to-be-tested voice collected by the voice input device;

performing signal preprocessing on the reference signal and the echo signal respectively; and

inputting the processed reference signal and the processed echo signal into a pre-trained time delay estimation model, to obtain a time difference between the reference signal and the echo signal output by the time delay estimation model, the time delay estimation model being used to represent a corresponding relationship between the reference signal, the echo signal and the time difference, wherein the time delay estimation model is configured to extract a feature of the reference signal and a feature of the echo signal, and is obtained by training operations based on long-term correlations between features of reference signals and features of echo signals; the time delay estimation model comprises: a convolutional neural network, a temporal convolutional network, and a fully connected layer, wherein the convolutional neural network, the temporal convolutional network and the fully connected layer are connected in sequence, the convolutional neural network is configured to extract and deeply fuse the feature of the reference signal and the feature of the echo signal, the temporal convolutional network is configured to learn the long-term correlation between the feature of the reference signal and the feature of the echo signal, and the fully connected layer is configured to extract the time delay between the reference signal and the echo signal.

8. The apparatus according to claim 7 , wherein, before acquiring the reference signal of the to-be-tested voice, the operations further comprise:

acquiring a plurality of constant-amplitude sweep-frequency signals;

inputting at least one sweep-frequency signal into the voice output device, and acquiring an output signal of the sweep-frequency signal in the voice input device;

detecting whether the voice output device and the voice input device are non-linear devices, based on the sweep-frequency signal and the output signal corresponding to the sweep-frequency signal; and

in response to determining that the voice output device and the voice input device are non-linear devices, acquiring the reference signal of the to-be-tested voice.

9. The apparatus according to claim 8 , wherein the operations further comprise:

in response to determining that the voice output device and the voice input device are not non-linear devices, calculating the time difference between the reference signal and the echo signal using a time delay estimation algorithm.

10. The apparatus according to claim 7 , wherein the time delay estimation model is obtained through training operations as follows:

acquiring a training sample, wherein the training sample comprises a sample signal and an echo signal corresponding to the sample signal; and

performing the training operations: inputting a sample signal and an echo signal corresponding to the sample signal into a first convolutional neural network, to obtain a feature vector of the sample signal and a feature vector of the echo signal corresponding to the sample signal; inputting the feature vector of the sample signal and the feature vector of the echo signal corresponding to the sample signal into a second convolutional neural network, to obtain correlation information of the sample signal and the echo signal corresponding to the sample signal within a preset time period; inputting the correlation information of the sample signal and the echo signal corresponding to the sample signal within the preset time period into a fully connected layer, to obtain a time difference between the sample signal and the echo signal corresponding to the sample signal; in response to determining that the first convolutional neural network, the second convolutional neural network, and the fully connected layer meet a training completion condition, using the first convolutional neural network, the second convolutional neural network, and the fully connected layer as the time delay estimation model.

11. The apparatus according to claim 7 , wherein the performing signal preprocessing on the reference signal and the echo signal respectively, comprises:

performing subband decomposition and catenation on the reference signal and the echo signal respectively; and

obtaining magnitude spectrograms of the catenated signals to obtain the processed reference signal and the processed echo signal.

12. The apparatus according to claim 7 , wherein the performing signal preprocessing on the reference signal and the echo signal respectively, comprises: performing frequency domain transformation on the reference signal and the echo signal respectively, to obtain the processed reference signal and the processed echo signal.

13. A non-transitory computer readable storage medium storing computer instructions, wherein, the computer instructions, when executed by a computer, cause the computer to perform operations, the operations comprising:

acquiring a reference signal of a to-be-tested voice, the reference signal being a signal output to a voice output device, wherein the voice output device outputs the to-be-tested voice after obtaining the reference signal;

receiving, from a voice input device, an echo signal of the to-be-tested voice, the echo signal being a signal of the to-be-tested voice collected by the voice input device;

performing signal preprocessing on the reference signal and the echo signal respectively; and

inputting the processed reference signal and the processed echo signal into a pre-trained time delay estimation model, to obtain a time difference between the reference signal and the echo signal output by the time delay estimation model, the time delay estimation model being used to represent a corresponding relationship between the reference signal, the echo signal and the time difference, wherein the time delay estimation model is configured to extract a feature of the reference signal and a feature of the echo signal, and is obtained by training operations based on long-term correlations between features of reference signals and features of echo signals; the time delay estimation model comprises: a convolutional neural network, a temporal convolutional network, and a fully connected layer, wherein the convolutional neural network, the temporal convolutional network and the fully connected layer are connected in sequence, the convolutional neural network is configured to extract and deeply fuse the feature of the reference signal and the feature of the echo signal, the temporal convolutional network is configured to learn the long-term correlation between the feature of the reference signal and the feature of the echo signal, and the fully connected layer is configured to extract the time delay between the reference signal and the echo signal.

14. The storage medium according to claim 13 , wherein, before acquiring the reference signal of the to-be-tested voice, the operations further comprise:

acquiring a plurality of constant-amplitude sweep-frequency signals;

inputting at least one sweep-frequency signal into the voice output device, and acquiring an output signal of the sweep-frequency signal in the voice input device;

detecting whether the voice output device and the voice input device are non-linear devices, based on the sweep-frequency signal and the output signal corresponding to the sweep-frequency signal; and

in response to determining that the voice output device and the voice input device are non-linear devices, acquiring the reference signal of the to-be-tested voice.

15. The storage medium according to claim 14 , wherein the operations further comprise:

in response to determining that the voice output device and the voice input device are not non-linear devices, calculating the time difference between the reference signal and the echo signal using a time delay estimation algorithm.

16. The storage medium according to claim 13 , wherein the time delay estimation model is obtained through training operations as follows:

acquiring a training sample, wherein the training sample comprises a sample signal and an echo signal corresponding to the sample signal; and

performing the training operations: inputting a sample signal and an echo signal corresponding to the sample signal into a first convolutional neural network, to obtain a feature vector of the sample signal and a feature vector of the echo signal corresponding to the sample signal; inputting the feature vector of the sample signal and the feature vector of the echo signal corresponding to the sample signal into a second convolutional neural network, to obtain correlation information of the sample signal and the echo signal corresponding to the sample signal within a preset time period; inputting the correlation information of the sample signal and the echo signal corresponding to the sample signal within the preset time period into a fully connected layer, to obtain a time difference between the sample signal and the echo signal corresponding to the sample signal; in response to determining that the first convolutional neural network, the second convolutional neural network, and the fully connected layer meet a training completion condition, using the first convolutional neural network, the second convolutional neural network, and the fully connected layer as the time delay estimation model.

17. The storage medium according to claim 13 , wherein the performing signal preprocessing on the reference signal and the echo signal respectively, comprises:

performing subband decomposition and catenation on the reference signal and the echo signal respectively; and

obtaining magnitude spectrograms of the catenated signals to obtain the processed reference signal and the processed echo signal.

18. The storage medium according to claim 13 , wherein the performing signal preprocessing on the reference signal and the echo signal respectively, comprises: performing frequency domain transformation on the reference signal and the echo signal respectively, to obtain the processed reference signal and the processed echo signal.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (37)