IP Library Granted Patent US 9,980,040
Granted Patent B2
US 9,980,040 · App. 15/441,793 · Granted May 22, 2018

Active speaker location detection

Inventors: Oliver Arthur Whyte (Cambridge, MA); Ross Cutler (Clyde Hill, WA); Avronil Bhattacharjee (Redmond, WA); Adarsh Prakash Murthy Kowdle (Redmond, WA); Adam Kirk (Seattle, WA); Stanley T. Birchfield (Sammamish, WA); Cha Zhang (Sammamish, WA)
Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
H04R1/406H04N7/142H04N7/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,980,040
App. No.
15/441,793
Granted
May 22, 2018
Kind
B2
Abstract

Various examples related to determining a location of an active participant are provided. In one example, image data of a room from an image capture device is received. First audio data from a first microphone array at the image capture device is received. Second audio data from a second microphone array spaced from the image capture device is received. Using a three dimensional model, a location of the second microphone array is determined. Using the first audio data, second audio data, location of the second microphone array, and an angular orientation of the second microphone array, an estimated location of the active participant is determined.

Claims (55)

1. A method for determining a location of an active participant, the method comprising:

from an image capture device, receiving image data of a room in which the active participant and at least one inactive participant are located;

from a first microphone array at the image capture device, receiving first audio data;

from a second microphone array spaced from the image capture device, receiving second audio data;

using a three dimensional model of at least a portion of the room, determining a location of the second microphone array; and

using at least the first audio data, the second audio data, the location of the second microphone array, and an angular orientation of the second microphone array, determining an estimated location of the active participant.

2. The method of claim 1 , further comprising:

computing a setting for the image capture device based on at least the estimated location of the active participant; and

outputting the setting to control the image capture device to highlight the active participant.

3. The method of claim 1 , wherein the image capture device comprises a color camera and the image data comprises color image data.

4. The method of claim 1 , wherein the image capture device comprises a depth camera and the image data comprises depth data.

5. The method of claim 1 , wherein the image data comprises signals corresponding to light emitted from a plurality of light sources of the second microphone array, and the method further comprises using the signals to determine the angular orientation of the second microphone array with respect to the image capture device.

6. The method of claim 5 , wherein the plurality of light sources are illuminated in a spatially-recognizable manner.

7. The method of claim 1 , further comprising:

receiving a signal from a magnetometer in the second microphone array; and

using the magnetometer signal, determining the angular orientation of the second microphone array.

8. The method of claim 1 , further comprising:

determining that at least one of the first microphone array and the second microphone array has moved; and

based on determining that at least one of the first microphone array and the second microphone array has moved, recomputing one or more of the location and the angular orientation of the second microphone array.

9. The method of claim 8 , wherein determining that at least one of the first microphone array and the second microphone array has moved comprises analyzing a signal received from one or more of an accelerometer in the first microphone array, a magnetometer in the first microphone array, an accelerometer in the second microphone array, and a magnetometer in the second microphone array.

10. The method of claim 1 , further comprising:

determining that the image data does not comprise image data of a plurality of light sources of the second microphone array; and

outputting a notification indicating that the second microphone array is occluded from view of the image capture device.

11. A video conferencing device, comprising:

an image capture device for capturing image data of a room in which an active participant and at least one inactive participant are located;

a first microphone array;

a processor; and

an active participant location program executable by the processor, the active participant location program configured to:

receive first audio data from the first microphone array;

receive second audio data from a second microphone array that is spaced from the image capture device;

using a three dimensional model of at least a portion of the room, determine a location of the second microphone array; and

using at least the first audio data, the second audio data, the location of the second microphone array, and an angular orientation of the second microphone array, determine an estimated location of the active participant.

12. The video conferencing device of claim 11 , wherein the active participant location program is configured to:

use at least the estimated location of the active participant to compute a setting for the image capture device; and

output the setting to control the image capture device to highlight the active participant.

13. The video conferencing device of claim 11 , wherein the image capture device comprises a color camera and the image data comprises color image data.

14. The video conferencing device of claim 11 , wherein the image data comprises signals corresponding to light emitted from a plurality of light sources of the second microphone array, and the active participant location program is configured to determine the angular orientation of the second microphone array using the signals.

15. The video conferencing device of claim 14 , wherein the plurality of light sources are illuminated in a spatially-recognizable manner.

16. The video conferencing device of claim 11 , wherein the active participant location program is configured to determine the angular orientation of the second microphone array using a signal received from a magnetometer in the second microphone array.

17. The video conferencing device of claim 11 , wherein the active participant location program is further configured to:

determine that the second microphone array has moved from a first location to a second location; and

based on determining that the second microphone array has moved, recompute one or more of the location and the angular orientation of the second microphone array.

18. The video conferencing device of claim 17 , wherein determining that the second microphone array has moved comprises receiving a signal from an accelerometer in the second microphone array.

19. The video conferencing device of claim 11 , wherein the active participant location program is further configured to:

determine that the image data does not comprise image data of a plurality of light sources of the second microphone array; and

output a notification indicating that the second microphone array is occluded from view of the image capture device.

20. A method for determining a location of an active participant, the method comprising:

from an image capture device, receiving image data of a room in which the active participant and at least one inactive participant are located;

from a first microphone array at the image capture device, receiving first audio data from the room;

from a second microphone array spaced from the image capture device, receiving second audio data;

using a three dimensional model of at least a portion of the room, determining a location of the second microphone array with respect to the image capture device;

determining an angular orientation of the second microphone array with respect to the image capture device by receiving light emitted from a plurality of light sources of the second microphone array;

using at least the first audio data, the second audio data, the location of the second microphone array, and the angular orientation of the second microphone array, determining an estimated three dimensional location of the active participant;

using the estimated location of the active participant to compute a setting for the image capture device; and

outputting the setting to control the image capture device to zoom into the active participant.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2017
From: WHYTE, OLIVER ARTHUR; CUTLER, ROSS; BHATTACHARJEE, AVRONIL; KOWDLE, ADARSH PRAKASH MURTHY; KIRK, ADAM; BIRCHFIELD, STANLEY T.; ZHANG, CHA
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 041371/0563 →
Continuity (2)
Continuation 14991847 · Jan 8, 2016
Related Publication 20170201825A1 · Jul 13, 2017