IP Library Granted Patent US 11,037,556
Granted Patent B2
US 11,037,556 · App. 16/037,945 · Granted Jun 15, 2021

Speech recognition for vehicle voice commands

Inventors: Ranjani Rangarajan (Farmington Hills, MI); John Edward Huber (Novi, MI); Leah N. Busch (Berkley, MI); Joshua Wheeler (Trenton, MI); Scott Andrew Amman (Milford, MI)
Assignee: Ford Global Technologies, LLC
G10L15/22B60K35/00G06F3/167B60K2370/148G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,037,556
App. No.
16/037,945
Granted
Jun 15, 2021
Kind
B2
Abstract

Method and apparatus are disclosed for speech recognition for vehicle voice commands. An example vehicle includes a microphone to collect a signal including a voice command, memory, and a controller. The controller is configured to determine an initial identification by feeding the signal into a first automatic speech recognition (ASR) engine and determine habits by feeding user history into a habits engine. The controller also is configured to identify the voice command by feeding the signal, the initial identification, and the habits into a second ASR engine. The controller also is configured to perform a vehicle function based on the voice command.

Claims (44)

1. A vehicle comprising:

a microphone to collect a signal including a voice command;

a memory; and

a controller configured to:

determine an initial identification of the voice command by feeding the signal into a first automatic speech recognition (ASR) engine, wherein the initial identification of the voice command is a final output of the first ASR engine, wherein to determine the initial identification comprises identifying, via an acoustic model of the first ASR engine, one or more phonemes of a dialect within the signal;

determine habits by feeding user history into a habits engine;

identify, based on determining that a confidence level associated with the initial identification is greater than a confidence threshold and a noise level associated with the initial identification is greater than a noise threshold, the voice command by feeding the signal, the initial identification, and the habits into a second ASR engine, wherein to identify the voice command comprises identifying, via a language model of the second ASR engine, one or more words within the signal by determining word probability distributions based on the one or more phonemes identified by the acoustic model; and

perform a vehicle function based on the voice command.

2. The vehicle of claim 1 , wherein the controller utilizes the second ASR engine to identify the voice command responsive to determining that the initial identification corresponds with the confidence level that is less than the confidence threshold.

3. The vehicle of claim 1 , wherein the controller identifies the initial identification as the voice command responsive to determining that:

the initial identification corresponds with the confidence level that is greater than the confidence threshold; and

the noise level of the signal is less than the noise threshold.

4. The vehicle of claim 1 , wherein the first ASR engine further includes:

the language model.

5. The vehicle of claim 1 , wherein the language model includes a deep neural network.

6. The vehicle of claim 1 , wherein the habits engine includes a pattern recognition algorithm.

7. The vehicle of claim 1 , further including one or more input devices, wherein the controller determines the user history based on user inputs received by the one or more input devices.

8. A system comprising:

a vehicle to operate based on a voice command, the vehicle includes a microphone to collect a signal including the voice command; and

a remote server, in communication with the vehicle, to:

determine an initial identification of the voice command via a first engine based on the signal, wherein the initial identification of the voice command is a final output of the first engine, wherein to determine the initial identification comprises identifying, via an acoustic model of the first engine, one or more phonemes of a dialect within the signal;

determine habits via a habits engine; and

identify, based on determining that a confidence level associated with the initial identification is greater than a confidence threshold and a noise level associated with the initial identification is greater than a noise threshold, the voice command for the vehicle via a second engine based on the signal, the initial identification, and the habits, wherein the first engine is different from the second engine, wherein to identify the voice command comprises identifying, via a language model of the second engine, one or more words within the signal by determining word probability distributions based on the one or more phonemes identified by the acoustic model.

9. The system of claim 8 , wherein the remote server utilizes the second engine to identify the voice command responsive to determining that the initial identification corresponds with the confidence level that is less than the confidence threshold.

10. The system of claim 8 , wherein the remote server identifies the initial identification as the voice command responsive to determining that:

the initial identification corresponds with the confidence level that is greater than the confidence threshold; and

the noise level of the signal is less than the noise threshold.

11. The system of claim 8 , wherein the first engine further includes:

the language model.

12. The system of claim 8 , wherein the language model includes a deep neural network.

13. The system of claim 8 , wherein the habits engine includes a pattern recognition algorithm.

14. The system of claim 8 , wherein the vehicle includes one or more input devices, wherein the remote server determines user history to be fed into the habits engine based on user inputs received by the one or more input devices.

15. A method comprising:

collecting, via a vehicle microphone, a signal that includes a voice command;

determining an initial identification of the voice command by feeding the signal into a first automatic speech recognition (ASR) engine, wherein the initial identification of the voice command is a final output of the first ASR engine, wherein determining the initial identification comprises identifying, via an acoustic model of the first ASR engine, one or more phonemes of a dialect within the signal;

determining habits by feeding user history into a habits engine;

identifying, based on determining that a confidence level associated with the initial identification is greater than a confidence threshold and a noise level associated with the initial identification is greater than a noise threshold, the voice command by feeding the signal, the initial identification, and the habits into a second ASR engine, wherein the first ASR engine is different from the second ASR engine, wherein identifying the voice command comprises identifying, via a language model of the second ASR engine, one or more words within the signal by determining word probability distributions based on the one or more phonemes identified by the acoustic model; and

performing, via a processor, a vehicle function based on the voice command.

16. The method of claim 15 , wherein the voice command is identified via the second ASR engine responsive to determining that the initial identification corresponds with the confidence level that is less than the confidence threshold.

17. The method of claim 15 , further including identifying the initial identification as the voice command responsive to determining that:

the confidence level that is greater than the confidence threshold; and

the noise level of the signal is less than the noise threshold.

18. The method of claim 15 , wherein the language model includes a deep neural network.

19. The method of claim 15 , wherein the habits engine includes a pattern recognition algorithm.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 6, 2018
From: RANGARAJAN, RANJANI; HUBER, JOHN EDWARD; BUSCH, LEAH N.; WHEELER, JOSHUA; AMMAN, SCOTT ANDREW
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 046558/0468 →
Continuity (1)
Related Publication 20200027452A1 · Jan 23, 2020
Cited By (4)
US 12,230,266 US 12,286,014 US 12,450,275 US 12,662,146