Granted Patent B2

US 12,412,591 · App. 18/279,475 · Granted Sep 9, 2025

Inventors: Haikuan Gao (Beijing, CN); Zhenyi Liu (Beijing, CN); Zhichao Wang (Beijing, CN); Jianyong Xuan (Beijing, CN); Risheng Xia (Beijing, CN)

Assignee: BEIJING HONOR DEVICE CO., LTD.

G10L21/0232G10L2021/02082G10L2021/02161

Abstract

Claims (58)

1. A voice processing method, applied to an electronic device, wherein the electronic device comprises n microphones, n is greater than or equal to 2, and the method comprises:

performing Fourier transform on voice signals picked up by the n microphones to obtain n channels of corresponding first frequency domain signals S, wherein each channel of first frequency domain signal S has M frequencies, and M is a quantity of transform points used when the Fourier transform is performed;

performing de-reverberation processing on the n channels of first frequency domain signals S to obtain n channels of second frequency domain signals S E , and performing noise reduction processing on the n channels of first frequency domain signals S to obtain n channels of third frequency domain signals S s ;

determining a first voice feature corresponding to M frequencies of a second frequency domain signal S Ei corresponding to a first frequency domain signal S i and a second voice feature corresponding to M frequencies of a third frequency domain signal S Si corresponding to the first frequency domain signal S i , and obtaining M target amplitude values corresponding to the first frequency domain signal S i based on the first voice feature, the second voice feature, the second frequency domain signal S Ei , and the third frequency domain signal S Si , wherein i=1, 2, . . . , or n, the first voice feature is used to represent a de-reverberation degree of the second frequency domain signal S Ei , and the second voice feature is used to represent a noise reduction degree of the third frequency domain signal S Si ; and

determining a fused frequency domain signal corresponding to the first frequency domain signal S i based on the M target amplitude values.

2. The method according to claim 1 , wherein the obtaining M target amplitude values corresponding to the first frequency domain signal S i based on the first voice feature, the second voice feature, the second frequency domain signal S Ei , and the third frequency domain signal S Si specifically comprises:

when it is determined that the first voice feature and the second voice feature that correspond to a frequency A i in the M frequencies meet a first preset condition, determining a first amplitude value corresponding to a frequency A i in the second frequency domain signal S Ei as a target amplitude value corresponding to the frequency A i , or determining the target amplitude value corresponding to the frequency A i based on the first amplitude value and a second amplitude value corresponding to a frequency A i in the third frequency domain signal S Si , wherein i=1, 2, . . . , or M; or

when it is determined that the first voice feature and the second voice feature that correspond to the frequency A i do not meet the first preset condition, determining the second amplitude value as the target amplitude value corresponding to the frequency A i .

3. The method according to claim 2 , wherein the determining the target amplitude value corresponding to the frequency A i based on the first amplitude value and a second amplitude value corresponding to a frequency A i in the third frequency domain signal S Si specifically comprises:

determining a first weighted amplitude value based on the first amplitude value corresponding to the frequency A i and a corresponding first weight, and determining a second weighted amplitude value based on the second amplitude value corresponding to the frequency A i and a corresponding second weight; and

determining a sum of the first weighted amplitude value and the second weighted amplitude value as the target amplitude value corresponding to the frequency A i .

4. The method according to claim 2 , wherein the first voice feature comprises a first dual-microphone correlation coefficient and a first frequency energy value, and the second voice feature comprises a second dual-microphone correlation coefficient and a second frequency energy value; and

the first dual-microphone correlation coefficient is used to represent a signal correlation degree between the second frequency domain signal S Ei and a second frequency domain signal S Et at corresponding frequencies, the second frequency domain signal S Et is any channel of second frequency domain signal S E other than the second frequency domain signal S Ei in the n channels of second frequency domain signals S E , the second dual-microphone correlation coefficient is used to represent a signal correlation degree between the third frequency domain signal S Si and a third frequency domain signal S St at corresponding frequencies, and the third frequency domain signal S St is a third frequency domain signal S s that is in the n channels of third frequency domain signals S s and that corresponds to a same first frequency domain signal as the second frequency domain signal S Et .

5. The method according to claim 4 , wherein the first preset condition is that the first dual-microphone correlation coefficient and the second dual-microphone correlation coefficient of the frequency A i meet a second preset condition, and the first frequency energy value and the second frequency energy value of the frequency A i meet a third preset condition.

6. The method according to claim 5 , wherein the second preset condition is that a first difference of the first dual-microphone correlation coefficient of the frequency A i minus the second dual-microphone correlation coefficient of the frequency A i is greater than a first threshold; and the third preset condition is that a second difference of the first frequency energy value of the frequency A i minus the second frequency energy value of the frequency A i is less than a second threshold.

7. The method according to claim 1 , wherein a de-reverberation processing method comprises a de-reverberation method based on a coherent-to-diffuse power ratio or a de-reverberation method based on a weighted prediction error.

8. The method according to claim 1 , wherein the method further comprises:

performing inverse Fourier transform on the fused frequency domain signal to obtain a fused voice signal.

9. The method according to claim 1 , wherein before the Fourier transform is performed on the voice signals, the method further comprises:

displaying a shooting interface, wherein the shooting interface comprises a first control;

detecting a first operation performed on the first control; and

in response to the first operation, performing, by the electronic device, video shooting to obtain a video that comprises the voice signals.

10. An electronic device, wherein the electronic device comprises:

n microphones, n is greater than or equal to 2;

one or more processors and

one or more memories; and the one or more memories are coupled to the one or more processors, the one or more memories are configured to store computer program code, the computer program code comprises computer instructions, and when the one or more processors execute the computer instructions, the electronic device is enabled to perform the following steps:

performing Fourier transform on voice signals picked up by the n microphones to obtain n channels of corresponding first frequency domain signals S, wherein each channel of first frequency domain signal S has M frequencies, and M is a quantity of transform points used when the Fourier transform is performed;

performing de-reverberation processing on the n channels of first frequency domain signals S to obtain n channels of second frequency domain signals S E , and performing noise reduction processing on the n channels of first frequency domain signals S to obtain n channels of third frequency domain signals S s ;

determining a first voice feature corresponding to M frequencies of a second frequency domain signal S Ei corresponding to a first frequency domain signal S i and a second voice feature corresponding to M frequencies of a third frequency domain signal S Si corresponding to the first frequency domain signal S i , and obtaining M target amplitude values corresponding to the first frequency domain signal S i based on the first voice feature, the second voice feature, the second frequency domain signal S Ei , and the third frequency domain signal S Si , wherein i=1, 2, . . . , or n, the first voice feature is used to represent a de-reverberation degree of the second frequency domain signal S Ei , and the second voice feature is used to represent a noise reduction degree of the third frequency domain signal S Si ; and

determining a fused frequency domain signal corresponding to the first frequency domain signal S i based on the M target amplitude values.

11. The electronic device according to claim 10 , wherein the obtaining M target amplitude values corresponding to the first frequency domain signal S i based on the first voice feature, the second voice feature, the second frequency domain signal S Ei , and the third frequency domain signal S Si specifically comprises:

when it is determined that the first voice feature and the second voice feature that correspond to a frequency A i in the M frequencies meet a first preset condition, determining a first amplitude value corresponding to a frequency A i in the second frequency domain signal S Ei as a target amplitude value corresponding to the frequency A i , or determining the target amplitude value corresponding to the frequency A i based on the first amplitude value and a second amplitude value corresponding to a frequency A i in the third frequency domain signal S Si , wherein i=1, 2, . . . , or M; or

when it is determined that the first voice feature and the second voice feature that correspond to the frequency A i do not meet the first preset condition, determining the second amplitude value as the target amplitude value corresponding to the frequency A i .

12. The electronic device according to claim 11 , wherein the determining the target amplitude value corresponding to the frequency A i based on the first amplitude value and a second amplitude value corresponding to a frequency A i in the third frequency domain signal S Si specifically comprises:

determining a first weighted amplitude value based on the first amplitude value corresponding to the frequency A i and a corresponding first weight, and determining a second weighted amplitude value based on the second amplitude value corresponding to the frequency A i and a corresponding second weight; and

determining a sum of the first weighted amplitude value and the second weighted amplitude value as the target amplitude value corresponding to the frequency A i .

13. The electronic device according to claim 11 , wherein the first voice feature comprises a first dual-microphone correlation coefficient and a first frequency energy value, and the second voice feature comprises a second dual-microphone correlation coefficient and a second frequency energy value; and

the first dual-microphone correlation coefficient is used to represent a signal correlation degree between the second frequency domain signal S Ei and a second frequency domain signal S Et at corresponding frequencies, the second frequency domain signal S Et is any channel of second frequency domain signal S E other than the second frequency domain signal S Ei in the n channels of second frequency domain signals S E , the second dual-microphone correlation coefficient is used to represent a signal correlation degree between the third frequency domain signal S Si and a third frequency domain signal S St at corresponding frequencies, and the third frequency domain signal S St is a third frequency domain signal S s that is in the n channels of third frequency domain signals S s and that corresponds to a same first frequency domain signal as the second frequency domain signal S Et .

14. The electronic device according to claim 13 , wherein the first preset condition is that the first dual-microphone correlation coefficient and the second dual-microphone correlation coefficient of the frequency A i meet a second preset condition, and the first frequency energy value and the second frequency energy value of the frequency A i meet a third preset condition.

15. The electronic device according to claim 14 , wherein the second preset condition is that a first difference of the first dual-microphone correlation coefficient of the frequency A i minus the second dual-microphone correlation coefficient of the frequency A i is greater than a first threshold; and the third preset condition is that a second difference of the first frequency energy value of the frequency A i minus the second frequency energy value of the frequency A i is less than a second threshold.

16. The electronic device according to claim 10 wherein a de-reverberation processing method comprises a de-reverberation method based on a coherent-to-diffuse power ratio or a de-reverberation method based on a weighted prediction error.

17. The electronic device according to claim 10 , wherein when the one or more processors execute the computer instructions, the electronic device is enabled to further perform the following steps:

performing inverse Fourier transform on the fused frequency domain signal to obtain a fused voice signal.

18. The electronic device according to claim 10 , wherein when the one or more processors execute the computer instructions, the electronic device is enabled to further perform the following steps:

before the Fourier transform is performed on the voice signals,

displaying a shooting interface, wherein the shooting interface comprises a first control;

detecting a first operation performed on the first control; and

in response to the first operation, performing, by the electronic device, video shooting to obtain a video that comprises the voice signals.

19. The electronic device according to claim 10 , wherein when the one or more processors execute the computer instructions, the electronic device is enabled to further perform the following steps:

before the Fourier transform is performed on the voice signals,

displaying a recording interface, wherein the recording interface comprises a second control;

detecting a second operation performed on the second control; and

in response to the second operation, performing, by the electronic device, recording to obtain the voice signals.

20. A non-transitory computer-readable storage medium, storing a computer program, wherein the computer program, when executed on an electronic device, causes the electronic device to perform following operations:

performing Fourier transform on voice signals picked up by the n microphones to obtain n channels of corresponding first frequency domain signals S, wherein each channel of first frequency domain signal S has M frequencies, and M is a quantity of transform points used when the Fourier transform is performed;

performing de-reverberation processing on the n channels of first frequency domain signals S to obtain n channels of second frequency domain signals S E , and performing noise reduction processing on the n channels of first frequency domain signals S to obtain n channels of third frequency domain signals S s ;

determining a first voice feature corresponding to M frequencies of a second frequency domain signal S Ei corresponding to a first frequency domain signal S i and a second voice feature corresponding to M frequencies of a third frequency domain signal S Si corresponding to the first frequency domain signal S i , and obtaining M target amplitude values corresponding to the first frequency domain signal S i based on the first voice feature, the second voice feature, the second frequency domain signal S Ei , and the third frequency domain signal S Si , wherein i=1, 2, . . . , or n, the first voice feature is used to represent a de-reverberation degree of the second frequency domain signal S Ei , and the second voice feature is used to represent a noise reduction degree of the third frequency domain signal S Si ; and

determining a fused frequency domain signal corresponding to the first frequency domain signal S i based on the M target amplitude values.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (45)