Voice command scrubbing
The invention is directed towards a an audio scrubbing system that allows for scrubbing recognized voice commands from audio data and replacing the recognized voice commands with environment audio data. Specifically, as a user captures video and audio data via a HMD, audio data captured by the HMD may be processed by an audio scrubbing module to identify voice commands in the audio data that are used for controlling the HMD. When a voice command is identified in the audio data, timestamps corresponding to the voice command may be determined. Filler audio data may then be generated to imitate the environment by processing at least a portion of the audio data by a neural network of a machine learning model. The filler audio data may then be used to replace the audio data corresponding to the identified voice commands, thereby scrubbing the voice command from the audio data.
1. A head-mounted computing device comprising:
a microphone;
one or more processors;
one or more memory devices storing programmable instructions thereon that, when executed by the one or more processors, cause the one or more processors to execute operations including:
receiving input data, wherein the received input data includes a recorded audio portion;
determining that the recorded audio portion includes an audible voice utterance that corresponds to a defined voice command, wherein the audible voice utterance has a first duration;
generating, via a machine learning model, a filler audio portion based on audio segments sampled from the recorded audio portion prior to and after the audible voice utterance, the filler audio portion having a second duration equal to the first duration and excluding the audible voice utterance; and
scrubbing the recorded audio portion based on the generated filler audio portion, the scrubbed recorded audio portion excluding the audible voice utterance.
2. The device of claim 1 , wherein scrubbing the audio portion further comprises identifying a voice pattern corresponding to the audible voice utterance.
3. The device of claim 2 , further comprising neutralizing the audible voice utterance that corresponds to the defined voice command based on combining a scrubbing voice pattern with the voice pattern corresponding to the audible voice utterance.
4. The device of claim 1 , further comprising a plurality of microphones; and
wherein scrubbing the audio portion further comprises parsing a set of audio signals from a plurality of microphones, wherein audio signals determined to be near a user mouth are muted.
5. The device of claim 1 , wherein based on determining that a boom arm of the device is rotated into a first position, activating, by an application of the device, a directional microphone configured to capture speech from a user wearing the device.
6. The device of claim 1 , further comprising identifying a portion of audio data corresponding to the defined voice command based on stored voice recognition characteristics.
7. The device of claim 1 , further comprising storing the scrubbed recorded audio portion including the filler audio portion and the recorded corresponding video portion to a memory of the head-mounted computing device.
8. A non-transitory computer storage medium storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:
receiving input data, wherein the received input data includes a recorded audio portion and a recorded corresponding video portion;
determining that the recorded audio portion includes an audible voice utterance that corresponds to a defined voice command, wherein the audible voice utterance has a first duration; and
generating, via a machine learning model, a filler audio portion based on audio segments sampled from the recorded audio portion prior to and after the audible voice utterance, the filler audio portion having a second duration equal to the first duration and excluding the audible voice utterance.
9. The computer storage medium of claim 8 , further comprising scrubbing the recorded audio portion based on the generated filler audio portion, the scrubbed recorded audio portion excluding the audible voice utterance and retaining environmental audio data corresponding to the audio portion.
10. The computer storage medium of claim 9 , wherein scrubbing the audio portion further comprises identifying a voice pattern corresponding to the audible voice utterance.
11. The computer storage medium of claim 10 , further comprising neutralizing the audible voice utterance that corresponds to the defined voice command based on combining a scrubbing voice pattern with the voice pattern corresponding to the audible voice utterance.
12. The computer storage medium of claim 8 , wherein scrubbing the audio portion further comprises parsing a set of audio signals from a plurality of microphones, wherein audio signals determined to be near a user mouth are muted.
13. The computer storage medium of claim 8 , wherein based on determining that a boom arm of the device is rotated into a first position, activating, by an application of the device, a directional microphone configured to capture speech from a user wearing the device.
14. The computer storage medium of claim 8 , further comprising identifying a portion of audio data corresponding to the defined voice command based on stored voice recognition characteristics.
15. A computer-implemented method comprising:
receiving input data, wherein the received input data includes an audio portion and a corresponding video portion;
determining that the audio portion includes an audible voice utterance that corresponds to a defined voice command;
excluding the audio portion from the input data based on the determination;
generating, by a machine learning model, a filler audio portion of a duration equal to a duration of the audible voice utterance and excluding the audible voice utterance; and
embedding the filler audio portion into the received input data based on a timestamp corresponding to the audible voice utterance.
16. The computer-implemented method of claim 15 , further comprising storing the scrubbed recorded audio portion including the filler audio portion and the recorded corresponding video portion to a memory of the head-mounted computing device.
17. The computer-implemented method of claim 15 , wherein excluding the audio portion further comprises identifying a voice pattern corresponding to the audible voice utterance.
18. The computer-implemented method of claim 17 , further comprising neutralizing the audible voice utterance that corresponds to the defined voice command based on combining a scrubbing voice pattern with the voice pattern corresponding to the audible voice utterance.
19. The computer-implemented method of claim 15 , wherein excluding the audio portion further comprises parsing a set of audio signals from a plurality of microphones, wherein audio signals determined to be near a user mouth are muted.
20. The computer-implemented method of claim 15 , further comprising:
wherein based on determining that a boom arm of the device is rotated into a first position, activating, by an application of the device, a directional microphone configured to capture speech from a user wearing the device.