Steering audio recordings in a surveillance system
A system may be configured to implement directional audio playback via digital signal processing on recorded audio. In some aspects, the system may include a processing device. Further, the processing device may be configured to select audio information for audio playback, the audio information including a plurality of audio captures of a microphone array, and determine directional instruction information for the audio playback, the directional instruction information identifying a virtual playback direction. In addition, the processing device may be further configured to generate a digital signal processing (DSP) plan based on the virtual playback direction, and apply the processing plan to the plurality of audio captures to produce the audio playback corresponding to the virtual playback direction.
1 . A system comprising:
processing device comprising:
a memory storing instructions thereon; and
at least one processor coupled to the memory and configured by the instructions to:
select audio information for audio playback, the audio information including a plurality of audio captures of a microphone array;
determine directional instruction information for the audio playback, the directional instruction information identifying a virtual playback direction based on a prediction location of a triggering event detected from the plurality of audio captures;
generate a digital signal processing (DSP) plan based on the virtual playback direction; and
apply the DSP plan to the plurality of audio captures to produce the audio playback corresponding to the virtual playback direction.
2 . The system of claim 1 , wherein to apply the DSP plan to the plurality of audio captures to produce the audio playback, the at least one processor is configured to apply a filter to one or more audio captures of the plurality of audio captures.
3 . The system of claim 1 , wherein to determine the directional instruction information for the audio playback, the at least one processor is configured to receive, via user input, the directional instruction information indicating pan movement during video playback associated with the audio playback.
4 . The system of claim 1 , wherein to determine the directional instruction information for the audio playback, the at least one processor is configured to receive, via first user input, the directional instruction information that is different from second user input indicating pan movement during video playback associated with the audio playback.
5 . The system of claim 1 , wherein to determine the directional instruction information for the audio playback, the at least one processor is configured to:
track a location of an object within one or more video frames; and
determine the directional instruction information for the audio playback based on the location of the object.
6 . The system of claim 5 , wherein to the at least one processor is further configured to receive a selection of the object.
7 . The system of claim 1 , wherein to determine the directional instruction information for the audio playback, the at least one processor is configured to:
identify, via a machine learning model, a predicted location of a desired audio output; and
determine the directional instruction information for the audio playback based on the predicted location.
8 . A method comprising:
selecting audio information for audio playback, the audio information including a plurality of audio captures of a microphone array;
determining directional instruction information for the audio playback, the directional instruction information identifying a virtual playback direction based on a prediction location of a triggering event detected from the plurality of audio captures;
generating a digital signal processing (DSP) plan based on the virtual playback direction; and
applying the DSP plan to the plurality of audio captures to produce the audio playback corresponding to the virtual playback direction.
9 . The method of claim 8 , wherein applying the DSP plan to the plurality of audio captures to produce the audio playback comprises applying a filter to one or more audio captures of the plurality of audio captures.
10 . The method of claim 8 , wherein determining the directional instruction information for the audio playback comprises receiving, via user input, the directional instruction information indicating pan movement during video playback associated with the audio playback.
11 . The method of claim 8 , wherein determining the directional instruction information for the audio playback comprises receiving, via first user input, the directional instruction information that is different from second user input indicating pan movement during video playback associated with the audio playback.
12 . The method of claim 8 , wherein determining the directional instruction information for the audio playback, comprises:
tracking a location of an object within one or more video frames; and
determining the directional instruction information for the audio playback based on the location of the object.
13 . The method of claim 8 , wherein determining the directional instruction information for the audio playback, comprises:
identifying, via a machine learning model, a predicted location of a desired audio output; and
determining the directional instruction information for the audio playback based on the predicted location.
14 . A non-transitory computer-readable device having instructions thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:
selecting audio information for audio playback, the audio information including a plurality of audio captures of a microphone array;
determining directional instruction information for the audio playback, the directional instruction information identifying a virtual playback direction based on a prediction location of a triggering event detected from the plurality of audio captures;
generating a digital signal processing (DSP) plan based on the virtual playback direction; and
applying the DSP plan to the plurality of audio captures to produce the audio playback corresponding to the virtual playback direction.
15 . The non-transitory computer-readable device of claim 14 , wherein applying the DSP plan to the plurality of audio captures to produce the audio playback comprises applying a filter to one or more audio captures of the plurality of audio captures.
16 . The non-transitory computer-readable device of claim 14 , wherein determining the directional instruction information for the audio playback comprises receiving, via user input, the directional instruction information indicating pan movement during video playback associated with the audio playback.
17 . The non-transitory computer-readable device of claim 14 , determining the directional instruction information for the audio playback comprises receiving, via first user input, the directional instruction information that is different from second user input indicating pan movement during video playback associated with the audio playback.
18 . The non-transitory computer-readable device of claim 14 , wherein determining the directional instruction information for the audio playback, comprises:
tracking a location of an object within one or more video frames; and
determining the directional instruction information for the audio playback based on the location of the object.
19 . The non-transitory computer-readable device of claim 14 , wherein determining the directional instruction information for the audio playback, comprises:
identifying, via a machine learning model, a predicted location of a desired audio output; and
determining the directional instruction information for the audio playback based on the predicted location.