Granted Patent B2

US 12,621,389 · App. 18/685,188 · Granted May 5, 2026

Inventors: Cheng Xue (Hangzhou City, CN); Weilong Huang (Hangzhou City, CN); Jinwei Feng (Winchester, MA)

Assignee: ZHEJIANG ALIBABA ROBOT CO., LTD.

H04M9/082G10L21/0208G10L2021/02082

Abstract

Claims (56)

1 . A conference terminal, comprising:

a loudspeaker;

at least one omnidirectional microphone set comprising at least two omnidirectional microphones;

a processor; and

a memory for storing a program which implements an echo cancellation method, the conference terminal, after being powered up and running the program for the echo cancellation method through the processor, performing the following steps:

determining a weight vector of a beamformer that enables the at least two omnidirectional microphones to form a dipole beam pattern, wherein the weight vector is configured to suppress an echo signal in a direction of the loudspeaker and enhance a sound signal in a target direction;

acquiring the sound signal through the at least two omnidirectional microphones, wherein the sound signal comprises the echo signal in the direction of the loudspeaker and the sound signal in the target direction; and

determining, for the omnidirectional microphone set, a weighted sum of at least two sound signals acquired by the at least two omnidirectional microphones based on the weight vector of the beamformer, wherein the weighted sum is an echo-canceled sound signal.

2 . The conference terminal according to claim 1 , wherein the number of the at least two omnidirectional microphones is two.

3 . The conference terminal according to claim 1 , wherein the number of the at least one omnidirectional microphone set is three, three omnidirectional microphone sets are arranged centered on the loudspeaker, the three omnidirectional microphone sets covering target sound sources in all directions.

4 . An echo cancellation method for a conference terminal, wherein the conference terminal comprises: a loudspeaker and at least one omnidirectional microphone set comprising at least two omnidirectional microphones;

the echo cancellation method comprising:

determining a weight vector of a beamformer that enables the at least two omnidirectional microphones to form a dipole beam pattern, wherein the weight vector is configured to suppress an echo signal in a direction of the loudspeaker and enhance a sound signal in a target direction;

acquiring the sound signal through the at least two omnidirectional microphones, wherein the sound signal comprises the echo signal in the direction of the loudspeaker and the sound signal in the target direction; and

determining, for the omnidirectional microphone set, a weighted sum of at least two sound signals acquired by the at least two omnidirectional microphones based on the weight vector of the beamformer, wherein the weighted sum is an echo-canceled sound signal.

5 . The method according to claim 4 , wherein determining the weight vector of the beamformer that enables the at least two omnidirectional microphones to form the dipole beam pattern, comprises:

determining a noise covariance matrix and a steering vector for the conference terminal; and

determining the weight vector based on the noise covariance matrix and the steering vector by means of a Minimum Variance Distortion-free Response (MVDR) beamforming algorithm.

6 . The method according to claim 5 , wherein the noise covariance matrix is determined by:

playing data of a preset sound when the conference terminal is started; and

determining a speech autocorrelation matrix as the noise covariance matrix based on the sound signal comprising the preset sound acquired by the at least two omnidirectional microphones.

7 . The method according to claim 6 , further comprising:

updating the autocorrelation matrix as an updated noise covariance matrix based on the sound signal comprising a conference sound acquired by the at least two omnidirectional microphones, if it is detected that it is mute in the target direction during the operation of the conference terminal.

8 . The method according to claim 4 , further comprising:

determining a signal-to-noise ratio of the at least two omnidirectional microphones if a movement of a target sound source is detected; and

selecting, according to the signal-to-noise ratio, the echo-canceled sound signal corresponding to the at least two omnidirectional microphones in a target omnidirectional microphone set.

9 . A sound pickup device, comprising:

a loudspeaker;

at least one omnidirectional microphone set comprising at least two omnidirectional microphones;

a processor; and

a memory for storing a program which implements the echo cancellation method according to claim 4 , the terminal being powered up and running the program for the echo cancellation method through the processor.

10 . The sound pickup device according to claim 9 , wherein the program implements the following steps:

determine a noise covariance matrix and a steering vector for the conference terminal; and

determine the weight vector based on the noise covariance matrix and the steering vector by means of a Minimum Variance Distortion-free Response (MVDR) beamforming algorithm.

11 . The sound pickup device according to claim 9 , wherein the program implements the following steps:

play data of a preset sound when the conference terminal is started; and

determine a speech autocorrelation matrix as the noise covariance matrix based on the sound signal comprising the preset sound acquired by the at least two omnidirectional microphones.

12 . The sound pickup device according to claim 9 , wherein the program implements the following steps:

update the autocorrelation matrix as an updated noise covariance matrix based on the sound signal comprising a conference sound acquired by the at least two omnidirectional microphones, if it is detected that it is mute in the target direction during the operation of the conference terminal.

13 . The sound pickup device according to claim 9 , wherein the program implements the following steps:

determine a signal-to-noise ratio of the at least two omnidirectional microphones if a movement of a target sound source is detected; and

select, according to the signal-to-noise ratio, the echo-canceled sound signal corresponding to the at least two omnidirectional microphones in a target omnidirectional microphone set.

14 . A computer program, comprising: computer-readable codes which, when run on a computing processing device, cause the computing processing device to execute the echo cancellation method according to claim 4 .

15 . A non-transitory computer-readable medium storing the computer program of claim 14 .

16 . The computer program according to claim 14 , wherein the computer-readable codes, when run on a computing processing device, cause the computing processing device to execute the following steps:

determine a noise covariance matrix and a steering vector for the conference terminal; and

determine the weight vector based on the noise covariance matrix and the steering vector by means of a Minimum Variance Distortion-free Response (MVDR) beamforming algorithm.

17 . The computer program according to claim 14 , wherein the computer-readable codes, when run on a computing processing device, cause the computing processing device to execute the following steps:

play data of a preset sound when the conference terminal is started; and

determine a speech autocorrelation matrix as the noise covariance matrix based on the sound signal comprising the preset sound acquired by the at least two omnidirectional microphones.

18 . The computer program according to claim 14 , wherein the computer-readable codes, when run on a computing processing device, cause the computing processing device to execute the following steps:

update the autocorrelation matrix as an updated noise covariance matrix based on the sound signal comprising a conference sound acquired by the at least two omnidirectional microphones, if it is detected that it is mute in the target direction during the operation of the conference terminal.

19 . The computer program according to claim 14 , wherein the computer-readable codes, when run on a computing processing device, cause the computing processing device to execute the following steps:

determine a signal-to-noise ratio of the at least two omnidirectional microphones if a movement of a target sound source is detected; and

select, according to the signal-to-noise ratio, the echo-canceled sound signal corresponding to the at least two omnidirectional microphones in a target omnidirectional microphone set.

20 . The non-transitory computer-readable medium according to claim 14 , wherein the non-transitory computer-readable medium stores the computer program.

Assignments (1)

Continuity (1)

References Cited (20)