Systems and methods for talker tracking and camera positioning in the presence of acoustic reflections
Systems and methods configured to generate talker coordinates for directing a camera towards an active talker in the presence of acoustic reflections are disclosed. One method comprises receiving sound location information for a detected audio source from a microphone; determining, based on the sound location information, a first set of coordinates representing an estimated talker location; determining, based on the sound location information and a height of the environment, a second set of coordinates representing a corrected talker location; calculating a weighted height coordinate based on a first height coordinate of the first set of coordinates, a second height coordinate of the second set of coordinates, and stored height coordinates from previously detected audio sources; and transmitting, to a camera, a third set of coordinates comprising the weighted height coordinate and representing a final talker location, to cause the camera to point its image capturing component towards the received location.
1 . A method performed by one or more processors in communication with each of a camera and at least one microphone disposed in an environment, the method comprising:
receiving, from the at least one microphone, sound location information for an audio source detected by the at least one microphone;
determining, based on the sound location information, a first set of coordinates representing an estimated talker location for the audio source;
determining, based on the sound location information and a height measurement of the environment, a second set of coordinates representing a corrected talker location for the audio source;
calculating a weighted height coordinate based on a first height coordinate of the first set of coordinates, a second height coordinate of the second set of coordinates, and stored height coordinates obtained for previously detected audio sources; and
transmitting, to the camera, a third set of coordinates comprising the weighted height coordinate and representing a final talker location for the audio source, wherein receipt of the third set of coordinates causes the camera to point an image capturing component of the camera towards the final talker location.
2 . The method of claim 1 , further comprising:
determining an amount of discrepancy between the first set of coordinates and the second set of coordinates; and
upon determining that the discrepancy exceeds a threshold, generating the third set of coordinates by replacing the height coordinate of the first set of coordinates with the weighted height coordinate.
3 . The method of claim 2 , wherein the threshold is configured to identify whether an acoustic reflection is present in the environment.
4 . The method of claim 1 , wherein calculating the weighted height coordinate comprises:
determining a first weight value for the first height coordinate based on the stored height coordinates;
determining a second weight value for the second height coordinate based on the stored height coordinates; and
calculating, using the first weight value and the second weight value, a weighted average of the first height coordinate and the second height coordinate.
5 . The method of claim 1 , wherein the camera is configured to point the image capturing component towards the final talker location by adjusting one or more of an angle, a tilt, a zoom, and a framing of the camera.
6 . The method of claim 1 , further comprising: determining the sound location information using an audio localization algorithm executed by an audio activity localizer.
7 . The method of claim 1 , wherein the height measurement comprises a height of the at least one microphone relative to a floor of the environment.
8 . A system comprising:
at least one microphone disposed in an environment and configured to determine sound location information for an audio source detected by the at least one microphone;
a camera disposed in the environment and comprising an image capturing component; and
one or more processors communicatively coupled to each of the at least one microphone and the camera, the one or more processors configured to:
receive the sound location information from the at least one microphone;
determine, based on the sound location information, a first set of coordinates representing an estimated talker location for the audio source;
determine, based on the sound location information and a height measurement of the environment, a second set of coordinates representing a corrected talker location for the audio source;
calculate a weighted height coordinate based on a first height coordinate of the first set of coordinates, a second height coordinate of the second set of coordinates, and stored height coordinates obtained for previously detected audio sources; and
transmit, to the camera, a third set of coordinates comprising the weighted height coordinate and representing a final talker location for the audio source,
wherein responsive to receiving the third set of coordinates, the camera is configured to point the image capturing component towards the final talker location.
9 . The system of claim 8 , wherein the one or more processors are further configured to:
determine an amount of discrepancy between the first set of coordinates and the second set of coordinates; and
upon determining that the discrepancy exceeds a threshold, generate the third set of coordinates by replacing the height coordinate of the first set of coordinates with the weighted height coordinate.
10 . The system of claim 9 , wherein the threshold is configured to identify whether an acoustic reflection is present in the environment.
11 . The system of claim 8 , wherein calculating the weighted height coordinate comprises:
determining a first weight value for the first height coordinate based on the stored height coordinates;
determining a second weight value for the second height coordinate based on the stored height coordinates; and
calculating, using the first weight value and the second weight value, a weighted average of the first height coordinate and the second height coordinate.
12 . The system of claim 8 , wherein the camera is configured to point the image capturing component towards the final talker location by adjusting one or more of an angle, a tilt, a zoom, and a framing of the camera.
13 . The system of claim 8 , further comprising an audio activity localizer configured to determine the sound location information using an audio localization algorithm executed by the audio activity localizer.
14 . The system of claim 8 , wherein the height measurement comprises a height of the at least one microphone relative to a floor of the environment.
15 . A non-transitory computer-readable storage medium comprising instructions that, when executed by one or more processors in communication with each of at least one microphone, and a camera, cause the one or more processors to perform the following:
receive sound location information for an audio source detected by the at least one microphone;
determine, based on the sound location information, a first set of coordinates representing an estimated talker location for the audio source;
determine, based on the sound location information and a height measurement of the environment, a second set of coordinates representing a corrected talker location for the audio source;
calculate a weighted height coordinate based on a first height coordinate of the first set of coordinates, a second height coordinate of the second set of coordinates, and stored height coordinates obtained for previously detected audio sources; and
transmit, to the camera, a third set of coordinates comprising the weighted height coordinate and representing a final talker location for the audio source, wherein receipt of the third set of coordinates causes the camera to point an image capturing component of the camera towards the final talker location.
16 . The non-transitory computer-readable storage medium of claim 15 , wherein the instructions further cause the one or more processors to:
determine an amount of discrepancy between the first set of coordinates and the second set of coordinates; and
upon determining that the discrepancy exceeds a threshold, generate the third set of coordinates by replacing the height coordinate of the first set of coordinates with the weighted height coordinate.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein the threshold is configured to identify whether an acoustic reflection is present in the environment.
18 . The non-transitory computer-readable storage medium of claim 15 , wherein calculating the weighted height coordinate comprises:
determining a first weight value for the first height coordinate based on the stored height coordinates;
determining a second weight value for the second height coordinate based on the stored height coordinates; and
calculating, using the first weight value and the second weight value, a weighted average of the first height coordinate and the second height coordinate.
19 . The non-transitory computer-readable storage medium of claim 15 , wherein the camera is configured to point the image capturing component towards the final talker location by adjusting one or more of an angle, a tilt, a zoom, and a framing of the camera.
20 . The non-transitory computer-readable storage medium of claim 15 , wherein the instructions further cause the one or more processors to determine the sound location information using an audio localization algorithm executed by an audio activity localizer.