IP Library Granted Patent US 10,939,202
Granted Patent B2
US 10,939,202 · App. 15/945,893 · Granted Mar 2, 2021

Controlling the direction of a microphone array beam in a video conferencing system

Inventors: Holger Stoltze (Billerica, MA); Ryo Tanaka (Concord, MA); Pascal Cleve (Sudbury, MA)
H04R3/005G10L21/0232G10L25/78H04R1/406H04R3/04G10L2021/02082H04N7/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,939,202
App. No.
15/945,893
Granted
Mar 2, 2021
Kind
B2
Abstract

A video conferencing system has a microphone array that operates to receive acoustic signals corresponding to voice activity and to determine that the signals are either from within a sound field of interest and from outside the sound field of interest, and if the signals are from outside the sound field of interest the system attenuates these signals by not steering a microphone array beam towards these signals.

Claims (26)

1. A method for controlling the direction of a microphone array beam, comprising:

determining that a first acoustic signal corresponding to voice activity generated by a first sound source arrives at the microphone array from a direction that is within a current sound field of interest, and controlling the operation of a beamforming function to steer the microphone array beam towards the direction of arrival of the first acoustic signal, wherein the beamforming function operates to only steer the microphone array beam toward the direction of an acoustic signal that arrives at the microphone array from within the current sound field of interest, and the current sound field of interest is variable and corresponds to a video camera field of view that is variable;

determining that a second acoustic signal corresponding to voice activity generated by a second sound source arrives at the microphone array from a direction that is outside the current sound field of interest during the time that the microphone array beam is steered to the direction of the acoustic signal generated by the first sound source; and

controlling the beamforming function to attenuate the second acoustic signal by preventing it from steering the microphone array beam forwards the second spend source during a time that it is determined that the first acoustic signal is no longer arriving at the microphone array.

2. The method of claim 1 , wherein the amount of attenuation is calculated or predetermined.

3. The method of claim 1 , wherein the camera field of view is variable.

4. The method of claim 1 , wherein the beamforming function is prevented from steering the microphone array beam towards the second sound source by disregarding the direction of arrival of the acoustic signal generated by the second sound source.

5. The method of claim 4 , wherein the direction of arrival of the acoustic signal generated by the second sound source does not correspond to a valid direction of arrival.

6. The method of claim 5 , wherein a valid direction of arrival is any direction of arrival that falls within the sound field of interest.

7. The method of claim 1 , wherein a degree to which sound energy associated with the acoustic signal generated by the second sound source is attenuated depends upon the value of the invalid direction of arrival of that acoustic signal.

8. A video conferencing system, comprising:

an array of two or more microphones disposed in a plane along a front surface of the video conferencing system that operates to receive acoustic signals;

acoustic echo cancellation functionality associated with each one of the microphones in the microphone array;

a voice activity detector for determining that the acoustic signals arriving at the microphone array correspond to voice activity;

a direction of arrival function that operates to determine a direction of arrival of the acoustic signal and to determine whether the direction of arrival of the acoustic signal is from a direction that is within or outside a current sound field of interest, wherein the current sound field of interest corresponds to a current video camera field of view; a microphone array beamformer; and a camera having a fixed or a variable field of view:

wherein a first acoustic signal is received by the microphone array that corresponds to voice activity and is determined to arrive from a direction that is within the sound field of interest and the beam former steering a microphone array beam towards the first acoustic signal, and a second acoustic signal is received by the microphone array that corresponds to voice activity and is determined to arrive from a direction that is outside the sound field of interest, and preventing the beamformer from steering the beam away from the direction of arrival of the first acoustic signal during a time that the second acoustic signal arrives at the microphone array whether or not the first acoustic signal generated by the first sound source is currently arriving at the microphone array whether or not the first acoustic signal generated by the first sound source is currently arriving at the microphone array.

9. The video conferencing system of claim 8 , further comprising calculating an amount of attenuation to apply to the second acoustic signal based on the direction of arrival of the signal, and applying the calculated amount of attenuation.

10. The method of claim 8 , wherein the camera field of view is variable.

11. The method of claim 8 , wherein the beamforming function is prevented from steering the microphone array beam towards the second sound source by disregarding the direction of arrival of the acoustic signal generated by the second sound source.

12. The method of claim 11 , wherein the direction of arrival of the acoustic signal generated by the second sound source does not correspond to a valid direction of arrival.

13. The method of claim 12 , wherein a valid direction of arrival is any direction of arrival that falls within the sound field of interest.

14. A method for controlling the direction of a microphone array beam, comprising:

determining that an acoustic signal corresponding to near-end voice activity generated by a sound source arrives at the microphone array from a direction that is outside a current sound field of interest, wherein the current sound field of interest corresponds to a current video camera field of view; and

preventing a microphone array beamforming function from steering a variable beam toward the direction of arrival of the acoustic signal while attenuating sound energy associated with the acoustic signal, and a degree to which the sound energy associated with the acoustic signal is attenuated is based upon the angle at which the acoustic signal arrives at the microphone array.

15. The method of claim 14 , wherein the amount of attenuation is calculated or predetermined.

16. The method of claim 14 , wherein the camera field of view is variable.

Assignments (8)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE RECORDED DATE AND THE REEL AND FRAME NUMBERS PREVIOUSLY RECORDED PREVIOUSLY RECORDED AT REEL: 063461 FRAME: 0046. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Nov 28, 2023
From: YAMAHA UNIFIED COMMUNICATIONS, INC.
To: YAMAMA CORPORATION
Reel/Frame 065692/0416 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ADDRESS OF ASSGNEE AND REEL & FRAME NUMBERS PREVIOUSLY RECORDED AT REEL: 063341 FRAME: 890. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Oct 31, 2023
From: REVOLABS, INC
To: YAMAHA UNIFIED COMMUNICATIONS, INC.
Reel/Frame 065593/0144 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 27, 2023
From: YAMAHA UNIFIED COMMUNICATIONS, INC.
To: YAMAHA CORPORATION
Reel/Frame 063461/0046 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 17, 2023
From: REVOLABS, INC.
To: YAMAHA UNIFIED COMMUNICATIONS, INC.
Reel/Frame 063341/0890 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 31, 2023
From: REVOLABS, INC.
To: YAMAHA UNIFIED COMMUNICATIONS, INC.
Reel/Frame 063193/0660 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2023
From: YAMAHA UNITED COMMUNICATIONS, INC.
To: YAMAHA CORPORATION, INC.
Reel/Frame 063096/0235 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2021
From: REVOLABS
To: YAMAHA UNITED COMMUNICATIONS
Reel/Frame 054931/0266 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 9, 2018
From: STOLTZE, HOLGER; CLEVE, PASCAL; TANAKA, RYO
To: REVOLABS, INC.
Reel/Frame 045483/0024 →
Continuity (1)
Related Publication 20190313187A1 · Oct 10, 2019
Cited By (1)
US 12,298,425