Utterance classifier

1. A computer-implemented method comprising:

receiving, by an automated assistant device of a classification system that includes (i) a classifier server, and (ii) the automated assistant device, audio data corresponding to an utterance;

obtaining, by the classifier server, a transcription of the utterance;

generating, by the classifier server, an acoustic feature vector corresponding to the audio data;

generating, by the classifier server, an acoustic feature vector corresponding to the transcription of the utterance;

providing, by the classifier server, (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the transcription of the utterance to a neural-network-based classifier that, based on a given acoustic feature vector corresponding to audio data and a given acoustic feature vector corresponding to a transcription of an utterance, is trained to output an indication of whether a given utterance associated with the given acoustic feature vector corresponding to the audio data is likely directed to an automated assistant or is likely not directed to an automated assistant;

determining, by the classifier server, that the classifier has output an indication that the utterance corresponding to the received audio data is likely directed to the automated assistant;

instructing, by the classifier server, the automated assistant device to generate a response to the utterance based at least on the indication that the utterance corresponding to the received audio data is likely directed to the automated assistant;

generating, by the automated assistant device and for output, the response to the utterance based on the instruction by the classifier server; and

providing, by the automated assistant device, the generated response for output.

2. The computer-implemented method of claim 1 , wherein providing (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the transcription of the utterance to the classifier, further comprises:

generating an input acoustic feature vector corresponding to a concatenation between (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the utterance; and

providing the concatenated input acoustic feature vector to the classifier.

3. The computer-implemented method of claim 2 , wherein generating the input acoustic feature vector corresponding to the concatenation between (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the transcription of the utterance, further comprises:

generating one or more numeric vectors converted from one or more words recognized in the transcription of the utterance; and

generating a matrix of numeric vectors comprising the one or more numeric vectors to provide to the classifier.

4. The computer-implemented method of claim 2 , further comprising:

receiving, from a speech recognizer at a word embedding model, recognizable text corresponding to the utterance;

generating, at the word embedding model, the transcription of the utterance from the recognizable text; and

providing, from the word embedding model, the transcription of the utterance to the classifier.

5. The computer-implemented method of claim 1 , wherein the acoustic feature vector corresponding to the audio data comprises acoustic mel-frequency cepstral coefficients (MFCCs) values.

6. The computer-implemented method of claim 4 , further comprising:

providing, by a max pooling layer of the word embedding model, the transcription of the utterance comprising embedding units;

providing, by an acoustic neural network, the acoustic feature vector corresponding to the audio data comprising audio units;

concatenating the embedding units and the audio units to a representation unit input; and

providing the representation unit input as input to the classifier.

7. The computer-implemented method of claim 1 , wherein the classifier server is in communication with the automated assistant device over a network.

8. A classification system that includes (i) a classifier server, and (ii) an automated assistant device, the classification system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving, by the automated assistant device, audio data corresponding to an utterance;

obtaining, by the classifier server, a transcription of the utterance;

generating, by the classifier server, an acoustic feature vector corresponding to the audio data;

generating, by the classifier server, an acoustic feature vector corresponding to the transcription of the utterance;

providing, by the classifier server, (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the transcription of the utterance to a neural-network-based classifier that, based on a given acoustic feature vector corresponding to audio data and a given acoustic feature vector corresponding to a transcription of an utterance, is trained to output an indication of whether a given utterance associated with the given acoustic feature vector corresponding to the audio data is likely directed to an automated assistant or is likely not directed to an automated assistant;

determining, by the classifier server, that the classifier has output an indication that the utterance corresponding to the received audio data is likely directed to the automated assistant;

instructing, by the classifier server, the automated assistant device to generate a response to the utterance based at least on the indication that the utterance corresponding to the received audio data is likely directed to the automated assistant;

generating, by the automated assistant device and for output, the response to the utterance based on the instruction by the classifier server; and

providing, by the automated assistant device, the generated response for output.

9. The system of claim 8 , wherein providing (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the transcription of the utterance to the classifier, further comprises:

generating an input acoustic feature vector corresponding to a concatenation between (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the utterance; and

providing the concatenated input acoustic feature vector to the classifier.

10. The system of claim 9 , wherein generating the input acoustic feature vector corresponding to the concatenation between (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the transcription of the utterance, further comprises:

generating one or more numeric vectors converted from one or more words recognized in the transcription of the utterance; and

generating a matrix of numeric vectors comprising the one or more numeric vectors to provide to the classifier.

11. The system of claim 9 , further comprising:

receiving, from a speech recognizer at a word embedding model, recognizable text corresponding to the utterance;

generating, at the word embedding model, the transcription of the utterance from the recognizable text; and

providing, from the word embedding model, the transcription of the utterance to the classifier.

12. The system of claim 11 , further comprising:

providing, by a max pooling layer of the word embedding model, the transcription of the utterance comprising embedding units;

providing, by an acoustic neural network, the acoustic feature vector corresponding to the audio data comprising audio units;

concatenating the embedding units and the audio units to a representation unit input; and

providing the representation unit input as input to the classifier.

13. The system of claim 8 , wherein the acoustic feature vector corresponding to the audio data comprises acoustic mel-frequency cepstral coefficients (MFCCs) values.

14. The system of claim 8 , wherein the classifier server is in communication with the automated assistant device over a network.

15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by an automated assistant device of a classification system that includes (i) a classifier server, and (ii) the automated assistant device, audio data corresponding to an utterance;

obtaining, by the classifier server, a transcription of the utterance;

generating, by the classifier server, an acoustic feature vector corresponding to of the audio data;

generating, by the classifier server, an acoustic feature vector corresponding to the transcription of the utterance;

providing, by the classifier server, (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the transcription of the utterance to a neural-network-based classifier that, based on a given acoustic feature vector corresponding to audio data and a given acoustic feature vector corresponding to a transcription of an utterance, is trained to output an indication of whether a given utterance associated with the given acoustic feature vector corresponding to the audio data is likely directed to an automated assistant or is likely not directed to an automated assistant;

determining, by the classifier server, that the classifier has output an indication that the utterance corresponding to the received audio data is likely directed to the automated assistant;

instructing, by the classifier server, the automated assistant device to generate a response to the utterance based at least on the indication that the utterance corresponding to the received audio data is likely directed to the automated assistant;

generating, by the automated assistant device and for output, the response to the utterance based on the instruction by the classifier server; and

providing, by the automated assistant device, the generated response for output.

16. The computer-readable medium of claim 15 , wherein providing (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the transcription of the utterance to the classifier, further comprises:

generating an input acoustic feature vector corresponding to a concatenation between (i) the acoustic feature vector corresponding to the audio data and (ii) the acoustic feature vector corresponding to the utterance; and

providing the concatenated input acoustic feature vector to the classifier.