Non-speech input to speech processing system
A system and method for a wearable device capable of detecting a wake gesture for purposes of capturing and forwarding audio data corresponding to a spoken utterance. The device may wake for purposes of capturing utterance audio data in response to a combination of a wake gesture and wakeword. The wake gesture may enable a wakeword detector. The device may also attempt to detect a wakeword utterance in a noisy environment. In response to determining the noisy environment, the device may receive motion data from a motion sensor; determining the motion data corresponds to a wake gesture, and send the audio data corresponding to an utterance to a remote device for processing. The device may also wake based on a combined confidence of wakeword and wake gesture detection.
1. A computer-implemented method of activating at least one wearable device to perform speech capturing to execute a spoken command, the method comprising:
receiving, from a microphone of the at least one wearable device, first audio data;
processing the first input audio data to determine a first confidence that the first audio data includes a representation of a wakeword;
receiving input motion data representing movements detected, wherein the input motion data includes sensor data comprising first motion sensor data corresponding to a first motion sensor of the at least one wearable device and second motion sensor data corresponding to a second motion sensor of the at least one wearable device;
determining that the sensor data indicates that the at least one device is moving about at least one of a vertical axis or a longitudinal axis;
processing the sensor data to determine a second confidence that the sensor data is similar to stored data corresponding to a gesture;
processing the first confidence and the second confidence using a trained model to determine that a wake command was intended;
determining second audio data corresponding to a second command;
sending the second audio data to a remote device for processing of the second command;
receiving, from the remote device, third audio data corresponding to the second command; and
outputting, from a speaker of the at least one wearable device, output audio corresponding to the third audio data.
2. The computer-implemented method of claim 1 , wherein:
the at least one wearable device comprising a first audio output speaker and a second audio output speaker,
includes the first motion sensor positioned proximate to the first audio output speaker and the second motion sensor positioned proximate to the second audio output speaker, and
the method further comprises:
determining the first sensor data represents the first motion sensor traveling a first direction about the longitudinal axis;
determining the second sensor data represents the second motion sensor traveling the first direction about the longitudinal axis; and
determining that the first sensor data and second sensor data correspond to a nod gesture.
3. The computer-implemented method of claim 1 , further comprising:
receiving second input motion data representing further movements detected by at least one of the first motion sensor or the second motion sensor of the at least one wearable device; and
determining the second input motion data corresponds to the second audio data, wherein the second audio data was stored on the at least one wearable device prior to receiving the first audio data.
4. A computer-implemented method comprising:
receiving audio data from at least one microphone of at least one device, the audio data corresponding to an utterance;
determining a signal quality metric of the audio data fails to satisfy a quality threshold corresponding to audio quality sufficient to process a spoken command;
receiving sensor data comprising first sensor data corresponding to a first sensor of the at least one device and second sensor data corresponding to a second sensor of the at least one device, wherein the sensor data indicates that the at least one device is moving about at least one of a vertical axis or a longitudinal axis;
processing the sensor data to determine a first confidence that the sensor data corresponds to a head motion;
determining that the first confidence satisfies a confidence threshold; and
in response to determining the signal quality metric fails to satisfy the quality threshold and the first confidence satisfies the confidence threshold, causing a command to be executed using first data.
5. The computer-implemented method of claim 4 , further comprising:
processing the audio data to determine a second confidence that the audio data includes a representation of a wakeword; and
processing the first confidence and the second confidence to determine that a wake command confidence threshold is satisfied.
6. The computer-implemented method of claim 4 , wherein the first data comprises at least a portion of the audio data.
7. The computer-implemented method of claim 4 , further comprising, after determining that the first confidence satisfies the confidence threshold:
enabling a wakeword component;
processing the audio data using the wakeword component to determine a second confidence that the audio data includes a representation of a wakeword;
determining that the second confidence satisfies a wakeword confidence threshold; and
sending the audio data to a remote device.
8. The computer-implemented method of claim 4 , wherein the at least one device comprises a first audio output speaker and a second audio output speaker and wherein receiving the first sensor data and the second sensor data comprises receiving the first sensor data from the first sensor positioned proximate to the first audio output speaker and receiving the second sensor data from the second sensor positioned proximate to the second audio output speaker.
9. The computer-implemented method of claim 8 , wherein the at least one wake command includes at least one of:
a head shake corresponding to a rotation of a head about the vertical axis,
a head nod corresponding to a movement of a head from back-to-front, or
a head tilt corresponding to a movement of a head about the longitudinal axis.
10. The computer-implemented method of claim 4 , further comprising, after determining the signal quality metric of the audio data fails to satisfy the confidence threshold, and before receiving the first sensor data and the second sensor data, outputting an indication wherein the indication is output by at least one of an audible output component, a lighting component, or a vibration component.
11. The computer-implemented method of claim 4 , further comprising:
determining stored data corresponding to the first sensor data and the second sensor data,
wherein the first data comprises the stored data.
12. The computer-implemented method of claim 4 , wherein the stored data comprises at least one of audio data or text data.
13. The computer-implemented method of claim 4 , wherein the first sensor is included as part of a first device configured to attach to an ear.
14. A system comprising:
a portion configured to be secured to a body part;
at least one microphone to detect input audio;
a first sensor;
a second sensor;
a communication component to communicate using a wireless network;
at least one processor; and
memory including instructions operable to be executed by the at least one processor to perform a set of actions to configure the system to:
receive audio data from the at least one microphone, the audio data corresponding to an utterance;
determine a signal quality metric of the audio data fails to satisfy a quality threshold corresponding to audio quality sufficient to process a spoken command;
receive sensor data corresponding to motion of the system, the sensor data comprising at least one of first sensor data from the first sensor and second sensor data from the second sensor;
process the sensor data to determine a first confidence that the sensor data corresponds to a head movement about at least one of a vertical axis or a longitudinal axis;
determine that the first confidence satisfies a confidence threshold; and
in response to determining the signal quality metric fails to satisfy the quality threshold and the first confidence satisfies the confidence threshold, causing a command to be executed using first data.
15. The system of claim 14 , wherein the instructions when executed further configure the system to:
process the audio data to determine a second confidence that the audio data includes a representation of a wakeword; and
process the first confidence and the second confidence to determine that a wake command confidence threshold is satisfied.
16. The system of claim 14 wherein the first data comprises at least a portion of the audio data.
17. The system of claim 14 , wherein the instructions when executed further configure the system to, after determining that the first confidence satifies the confidence threshold:
enable a wakeword component;
process the audio data using the wakeword component to determine a second confidence that the audio data includes a representation of a wakeword;
determine that the second confidence satisfies a wakeword confidence threshold; and
sending the audio data to a remote device.
18. The system of claim 14 , wherein the at least one speaker comprises a first audio output speaker and a second audio output speaker and the at least one sensor comprises a first sensor positioned proximate to the first audio output speaker and the second sensor positioned proximate to the second audio output speaker.
19. The system of claim 18 , wherein the head movement includes at least one of:
a head shake corresponding to a rotation of a head about the vertical axis,
a head nod corresponding to a movement of a head from back-to-front, or
a head tilt corresponding to a movement of a head about the longitudinal axis.
20. The system of claim 14 , wherein the instructions when executed further configure the system to, after determining the signal quality metric of the audio data fails to satisfy the confidence threshold, and before receiving the first sensor data and the second sensor data, output an indication wherein the indication is output by at least one of the at least one speaker, a lighting component, or a vibration component.
21. The system of claim 14 , wherein:
the instructions when executed further configure the system to determine stored data corresponding to the first sensor data and the second sensor data, and the first data comprises the stored data.
22. The system of claim 14 , wherein the instructions when executed further configure the system to:
determine the sensor data comprising at least one of first sensor data from the first sensor and the second sensor data from the second sensor correspond to the first sensor or the second sensor traveling in a first direction about the longitudinal axis.
23. The system of claim 14 , further comprising:
a first device configured to attach to an ear, the first device comprising the first sensor.