IP Library Granted Patent US 11,997,454
Granted Patent B1
US 11,997,454 · App. 17/368,509 · Granted May 28, 2024

Power efficient acoustic tracking of sound sources

Inventor: Andrew Lovitt (Redmond, WA)
Assignee: META PLATFORMS TECHNOLOGIES, LLC
H04R1/406H04R3/005G02B27/0176H04R2201/401
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,997,454
App. No.
17/368,509
Granted
May 28, 2024
Kind
B1
Abstract

A method of tracking sound sources in an environment by monitoring the local area with a primary sensor system comprising a microphone array. The location of sound sources in the local area are tracked using the monitored sound. If an ambiguity is determined in the tracked location of the sound source, a secondary sensor system is activated. The secondary sensor system has a larger power draw than the primary sensor system. The secondary sensor system determines the updated location of the sound source. The tracked location of the sound sources is updated to be the updated location. Once the location is updated, the secondary sensor system is deactivated.

Claims (66)

1. A method comprising:

monitoring, via primary sensor system, sound in a local area, wherein the primary sensor system is a microphone array;

tracking a location of a sound source in the local area using the monitored sound; and

responsive to determining an ambiguity condition in the tracked location:

activating a secondary sensor system that has a larger power draw than the microphone array;

determining an updated location of the sound source using the secondary sensor system;

updating the tracked location to be the updated location; and

deactivating the secondary sensor system.

2. The method of claim 1 further comprising:

updating a model with the tracked location; and

wherein updating the tracked location to be the updated location comprises:

updating the tracked location in the model to be the updated location of the sound source.

3. The method of claim 1 wherein the secondary sensor system includes at least one of a camera, a depth sensing system, or a wireless transceiver.

4. The method of claim 1 , wherein the ambiguity condition is a threshold period of time passing with sound from the sound source being below a threshold value.

5. The method of claim 1 , wherein the ambiguity condition is sound occurring within the local area at least a threshold distance from an expected location of the sound source.

6. The method of claim 1 , wherein the ambiguity condition is two sources of sound being located within a threshold distance from each other.

7. The method of claim 1 further comprising:

receiving, from an external device, a location request;

determining a location of an audio system that includes the primary sensor system and the secondary sensor system; and

providing the location to the external device.

8. The method of claim 1 , wherein the sound source is associated with a personal device, and the secondary sensor system is a wireless transceiver that communicatively couples to an external device, the method further comprising:

sending, via the wireless transceiver, a request for the location of the external device, wherein a position sensor on the external device determines location of the external device;

receiving, via the wireless transceiver, the determined location of the external device; and

wherein determining the updated location of the sound source using the secondary sensor system comprises updating a model with the determined location of the external device.

9. The method of claim 1 further comprising:

tracking a location of a second sound source using the primary sensor system concurrently to tracking the location of the sound source using the secondary sensor system,

wherein the second sound source is outside of an area of detection of the secondary sensor system.

10. The method of claim 1 , wherein determining the updated location of the sound source using the secondary sensor system further comprises:

determining that a location of the sound source determined by the secondary sensor system does not resolve the ambiguity condition;

activating an additional secondary sensor system that has a larger power draw than the primary sensor system; and

determining a location of the sound source using the additional secondary sensor system, wherein the determined location is the updated location.

11. A system comprising:

a primary sensor system comprising a microphone array configured to monitor sound in a local area;

a secondary sensor system configured to monitor at least a portion of the local area; and

a controller configured to:

track a location of a sound source using the monitored sound; and

in response to an ambiguity condition in the tracked location:

activate the secondary sensor system, wherein the secondary sensor system has a larger power draw than microphone array;

determine an updated location of the sound source using the secondary sensor system;

update the tracked location to be the updated location; and

deactivate the secondary sensor system.

12. The system of claim 11 wherein the controller is further configured to:

update a model with the tracked location; and

update the tracked location in the model to be the updated location of the sound source.

13. The system of claim 11 wherein the secondary sensor system is at least one of a camera, a depth sensing system, and a wireless transceiver.

14. The system of claim 11 , wherein the ambiguity condition is a threshold of period of time passing with sound from the sound source being below a threshold value.

15. The system of claim 11 , wherein the ambiguity condition is sound occurring within the local area at least a threshold distance from an expected location of the sound source.

16. The system of claim 11 , wherein the ambiguity condition is two sources of sound being located within a threshold distance of each other.

17. The system of claim 11 further configured to:

receive, from an external device, a location request;

determine a location of an audio system that includes the primary sensor system and the secondary sensor system; and

provide the location to the external device.

18. The system of claim 11 , wherein the sound source is associated with a personal device, and the secondary sensor system is a wireless transceiver that communicatively couples to an external device, the system further configured to:

send, via the wireless transceiver, a request for the location of the external device, wherein a position sensor on the external device determines location of the external device;

receive, via the wireless transceiver, the determined location of the external device; and

update a model with the determined location of the external device.

19. The system of claim 11 further configured to:

monitor a second sound source with the primary sensor system concurrently to monitoring a first sound source with the secondary sensor system.

20. A non-transitory computer readable medium configured to store program code instructions, which, when executed by a processor, cause an audio system to perform steps comprising:

monitoring, via a primary sensor system of the audio system, sound in a local area, wherein the primary sensor system is a microphone array;

tracking a location of a sound source in the local area using the monitored sound; and

responsive to determining an ambiguity condition in the tracked location:

activating a secondary sensor system that has a larger power draw than the microphone array;

determining an updated location of the sound source using the secondary sensor system;

updating the tracked location to be the updated location; and

deactivating the secondary sensor system.

Assignments (2)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060314/0965 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2021
From: LOVITT, ANDREW
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 056843/0501 →