Full-duplex adaptive echo cancellation in a conference endpoint

1. A method comprising:

at an endpoint device including one or more microphones:

converting sound to one or more microphone signals with the one or more microphones;

transforming the one or more microphone signals into far-field beam signals representative of far-field beams and near-field beam signals representative of near-field beams;

deriving a far-field noise level estimate from the far-field beam signals;

synthesizing far-field comfort noise based on the far-field noise level estimate, and applying the far-field comfort noise to the far-field beam signals to produce far-field output signals;

canceling echo in the near-field beam signals;

performing near-field non-linear processing (NLP) on the near-field beam signals to reduce residual echo in the near-field beam signals;

generating a near-field NLP gain based on the near-field NLP;

synthesizing near-field comfort noise based on both the far-field noise level estimate and the near-field NLP gain so that the near-field comfort noise is similar to the far-field comfort noise, and applying the near-field comfort noise to the near-field beam signals to produce near-field output signals;

selecting either the near-field output signals or the far-field output signals as output signals based on a selection criterion; and

transmitting a signal that is based on the output signals.

2. The method of claim 1 , further comprising:

canceling echo in the far-field beam signals, wherein the selecting includes:

determining whether the echo in the near-field beam signals or the echo in the far-field beam signals is above a threshold;

when the echo in the near-field beam signals or the echo in the far-field beam signals is above the threshold, selecting the near-field output signals; and

when neither the echo in the near-field beam signals nor the echo in the far-field beam signals is above the threshold, selecting the far-field output signals as the output signals.

3. The method of claim 2 , wherein the endpoint device includes a loudspeaker, and the method further comprises:

driving the loudspeaker with a loudspeaker signal; and

performing the canceling the echo in the far-field beam signals and the canceling the echo in the near-field beam signals using the loudspeaker signal.

4. The method of claim 1 , further comprising:

removing noise from the far-field beam signals based on the far-field noise level estimate.

5. The method of claim 4 , further comprising:

removing noise from the near-field beam signals.

6. The method of claim 1 ,

wherein the selection criterion includes a signal-to-noise ratio.

7. The method of claim 1 , further comprising:

canceling echo in the far-field beam signals; and

performing far-field NLP on the far-field beam signals to reduce residual echo in the far-field beam signals, and generating a far-field NLP gain based on the far-field NLP, wherein the synthesizing the far-field comfort noise includes synthesizing the far-field comfort noise based on both the far-field noise level estimate and the far-field NLP gain.

8. The method of claim 1 , further comprising:

dereverberating the far-field beam signals; and

dereverberating the near-field beam signals.

9. An apparatus comprising:

one or more microphones to convert sound to one or more microphone signals;

a network interface unit to communicate with a network; and

a processor coupled to the one or more microphones and the network interface unit, wherein the processor is configured to perform:

transforming the one or more microphone signals into far-field beam signals representative of far-field beams and near-field beam signals representative of near-field beams;

deriving a far-field noise level estimate from the far-field beam signals;

synthesizing far-field comfort noise based on the far-field noise level estimate, and applying the far-field comfort noise to the far-field beam signals to produce far-field output signals;

canceling echo in the near-field beam signals;

performing near-field non-linear processing (NLP) on the near-field beam signals to reduce residual echo in the near-field beam signals;

generating a near-field NLP gain based on the near-field NLP;

synthesizing near-field comfort noise based on both the far-field noise level estimate and the near-field NLP gain so that the near-field comfort noise is similar to the far-field comfort noise, and applying the near-field comfort noise to the near-field beam signals to produce near-field output signals;

selecting either the near-field output signals or the far-field output signals as output signals based on a selection criterion; and

transmitting a signal that is based on the output signals.

10. The apparatus of claim 9 , wherein the processor is further configured to perform:

canceling echo in the far-field beam signals, wherein the processor is configured to perform the selecting by:

determining whether the echo in the near-field beam signals or the echo in the far-field beam signals is above a threshold;

when the echo in the near-field beam signals or the echo in the far-field beam signals is above the threshold, selecting the near-field output signals; and

when neither the echo in the near-field beam signals nor the echo in the far-field beam signals is above the threshold, selecting the far-field output signals as the output signals.

11. The apparatus of claim 10 , further comprising a loudspeaker, wherein the processor is further configured to perform:

driving the loudspeaker with a loudspeaker signal,

wherein the processor is configured to perform the canceling the echo in the far-field beam signals and the canceling the echo in the near-field beam signals using the loudspeaker signal.

12. The apparatus of claim 9 , wherein the processor is further configured to perform:

removing noise from the far-field beam signals based on the far-field noise level estimate.

13. The apparatus of claim 12 , wherein the processor is further configured to perform:

removing noise from the near-field beam signals.

14. The apparatus of claim 9 , wherein the selection criterion includes a signal-to-noise ratio.

15. The apparatus of claim 9 , wherein the processor is further configured to perform:

canceling echo in the far-field beam signals; and

performing far-field NLP on the far-field beam signals to reduce residual echo in the far-field beam signals, and generating a far-field NLP gain based on the far-field NLP, wherein the processor is configured to perform the synthesizing the far-field comfort noise based on both the far-field noise level estimate and the far-field NLP gain.

16. The apparatus of claim 9 , wherein the processor is further configured to perform:

dereverberating the far-field beam signals; and

dereverberating the near-field beam signals.

17. A non-transitory processor readable medium storing instructions that, when executed by a processor of an endpoint device including one or more microphones, causes the processor to perform:

converting sound to one or more microphone signals with the one or more microphones;

transforming the one or more microphone signals into far-field beam signals representative of far-field beams and near-field beam signals representative of near-field beams;

deriving a far-field noise level estimate from the far-field beam signals;

canceling echo in the far-field beam signals;

performing far-field non-linear processing (NLP) on the far-field beam signals to reduce residual echo in the far-field beam signals;

generating a far-field NLP gain based on the far-field NLP;

synthesizing far-field comfort noise based on both the far-field noise level estimate and the far-field NLP gain, and applying the far-field comfort noise to the far-field beam signals to produce far-field output signals;

synthesizing near-field comfort noise from the far-field noise level estimate so that the near-field comfort noise is similar to the far-field comfort noise, and applying the near-field comfort noise to the near-field beam signals to produce near-field output signals;

selecting either the near-field output signals or the far-field output signals as output signals based on a selection criterion; and

transmitting a signal that is based on the output signals.

18. The non-transitory processor readable medium of claim 17 , further comprising instructions to cause the processor to perform:

canceling echo in the near-field beam signals, wherein the selecting includes:

determining whether the echo in the near-field beam signals or the echo in the far-field beam signals is above a threshold;

when the echo in the near-field beam signals or the echo in the far-field beam signals is above the threshold, selecting the near-field output signals; and

when neither the echo in the near-field beam signals nor the echo in the far-field beam signals is above the threshold, selecting the far-field output signals as the output signals.

19. The non-transitory processor readable medium of claim 17 , further comprising instructions to cause the processor to perform:

canceling echo in the near-field beam signals; and

performing NLP on the near-field beam signals to reduce residual echo in the near-field beam signals, and generating a near-field NLP gain based on the near-field NLP, wherein the instructions to cause the processor to perform the synthesizing the near-field comfort noise include instructions to cause the processor to perform synthesizing the near-field comfort noise based on both the far-field noise level estimate from the far-field beam signals and the near-field NLP.

20. The non-transitory processor readable medium of claim 17 ,

wherein the selection criterion includes a signal-to-noise ratio.