IP Library Granted Patent US 12662157
Granted Patent B2
US 12662157 · App. 18/467,917 · Granted Jun 23, 2026

Voice controlled autonomous driving system

Inventors: Rouhollah Sayed Jafari (Troy, MI); Alireza Esna Ashari Esfahani (San Bruno, CA); Upali P. Mudalige (Rochester Hills, MI)
Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
B60W60/001G10L15/16G10L15/22B60W2540/21G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12662157
App. No.
18/467,917
Granted
Jun 23, 2026
Kind
B2
Abstract

A voice controlled autonomous driving system for determining a trajectory of a vehicle based on a voice command includes one or more controllers executing instructions to receive an electrical signal representative of the voice command, where the voice command indicates a driving maneuver. The controllers convert the electrical signal into a text-based command, classify the text-based command into one of a plurality of pre-compiled driving maneuvers based on one or more machine learning algorithms, and classify the pre-compiled driving maneuver as either a route-based maneuver or a behavior-based maneuver, and in response to determining the precompiled driving maneuver is a route-based driving maneuver, identify one or more relevant lane segments of the map data affected by the pre-compiled driving maneuver and determine an updated route plan by updating a current route plan the vehicle is presently following to include the one or more relevant lane segments.

Claims (54)

1 . A voice controlled autonomous driving system for determining a trajectory of a vehicle based on a voice command, the voice controlled autonomous driving system comprising:

one or more controllers executing instructions to:

receive an electrical signal representative of the voice command generated by an occupant of the vehicle, wherein the voice command indicates a driving maneuver that alters motion of the vehicle;

convert the electrical signal into a text-based command based on one or more speech recognition algorithms;

classify the text-based command into one of a plurality of pre-compiled driving maneuvers based on one or more machine learning algorithms, wherein the pre-compiled driving maneuvers represent potential driving maneuvers that the vehicle has the potential to execute, and wherein the pre-compiled driving maneuvers include at least the following: take the next exit, to make a lane change, and to pass another vehicle;

classify the pre-compiled driving maneuver as either a route-based driving maneuver or a behavior-based driving maneuver;

in response to classifying the pre-compiled driving maneuver as the route-based driving maneuver, compare the pre-compiled driving maneuver with map data to identify one or more relevant lane segments of the map data affected by the pre-compiled driving maneuver;

determine an updated route plan by updating a current route plan the vehicle is presently following to include the one or more relevant lane segments, wherein the trajectory of the vehicle is determined based on the updated route plan; and

instruct the vehicle to perform at least one of accelerating, braking, and steering to follow the trajectory.

2 . The voice controlled autonomous driving system of claim 1 , wherein the one or more controllers execute instructions to:

in response to classifying the pre-compiled driving maneuver as a behavior-based driving maneuver, compare the pre-compiled driving maneuver with a plurality of behavior planner parameters to determine one or more updated behavior planner parameters.

3 . The voice controlled autonomous driving system of claim 2 , wherein the one or more controllers execute instructions to:

determine an updated goal state of the vehicle based on the one or more updated behavior planner parameters.

4 . The voice controlled autonomous driving system of claim 3 , wherein the updated goal state indicates a driving maneuver, the one or more updated behavior planner parameters, and the behavior planner parameters that are followed to execute the pre-compiled driving maneuver and the trajectory of the vehicle is determined based on the updated goal state.

5 . The voice controlled autonomous driving system of claim 1 , wherein a confidence score indicating a confidence of the one or more machine learning algorithms that classify the text-based command is associated with the pre-compiled driving maneuver.

6 . The voice controlled autonomous driving system of claim 5 , further comprising a human-machine interface (HMI) in electronic communication with the one or more controllers.

7 . The voice controlled autonomous driving system of claim 6 , wherein the one or more controllers execute instructions to:

compare the confidence score associated with the pre-compiled driving maneuver with a confidence threshold value; and

in response to determining the confidence score associated with the pre-compiled driving maneuver is less than the confidence threshold value, instruct the HMI to generate a message indicating the voice controlled autonomous driving system is unable to execute the voice command.

8 . The voice controlled autonomous driving system of claim 7 , wherein the one or more controllers execute instructions to:

in response to determining the confidence score associated with the pre-compiled driving maneuver is greater than or equal to the confidence threshold value, compare the pre-compiled driving maneuver with a set of traffic rules associated with a specific location of the vehicle; and

in response to determining the pre-compiled driving maneuver is authorized based on the set of traffic rules, compare the pre-compiled driving maneuver with a plurality of potential active voice commands stored in memory to determine the presence of a conflict between the pre-compiled driving maneuver and the one or the potential active voice commands.

9 . The voice controlled autonomous driving system of claim 8 , wherein the one or more controllers execute instructions to:

in response to determining the pre-compiled driving maneuver conflicts with one of the potential active voice commands, instruct the HMI to generate a message asking the occupant to select either the pre-compiled driving maneuver or the active voice command that conflicts with the pre-compiled driving maneuver.

10 . The voice controlled autonomous driving system of claim 8 , wherein the pre-compiled driving maneuver is classified as either the route-based driving maneuver or the behavior-based driving maneuver, in response to determining an absence of the conflict.

11 . The voice controlled autonomous driving system of claim 1 , further comprising a microphone in electronic communication with the one or more controllers, wherein the microphone converts the voice command into the electrical signal representative of the voice command.

12 . The voice controlled autonomous driving system of claim 1 , wherein the route-based driving maneuver represents a maneuver that alters a current route plan the vehicle is following.

13 . The voice controlled autonomous driving system of claim 1 , wherein the behavior-based driving maneuver represents a maneuver that governs a behavior of the vehicle but does not alter a current route plan the vehicle is presently following.

14 . A method for determining a trajectory of a vehicle based on a voice command by a voice controlled autonomous driving system, the method comprising:

receiving, by one or more controllers, an electrical signal representative of the voice command generated by an occupant of the vehicle, wherein the voice command indicates a driving maneuver that alters motion of the vehicle;

converting, by the one or more controllers, the electrical signal into a text-based command based on one or more speech recognition algorithms;

classifying, by the one or more controllers, the text-based command into one of a plurality of pre-compiled driving maneuvers based on one or more machine learning algorithms, wherein the pre-compiled driving maneuvers represent potential driving maneuvers that the vehicle has the potential to execute, and wherein the pre-compiled driving maneuvers include at least the following: take the next exit, to make a lane change, and to pass another vehicle;

classifying the pre-compiled driving maneuver as either a route-based driving maneuver or a behavior-based driving maneuver;

in response to classifying the pre-compiled driving maneuver as the route-based driving maneuver, comparing the pre-compiled driving maneuver with map data to identify one or more relevant lane segments of the map data affected by the pre-compiled driving maneuver;

determining an updated route plan by updating a current route plan the vehicle is presently following to include the one or more relevant lane segments, wherein the trajectory of the vehicle is determined based on the updated route plan; and

instruct the vehicle to perform at least one of accelerating, braking, and steering to follow the trajectory.

15 . A voice controlled autonomous driving system for determining a trajectory of a vehicle based on a voice command, the voice controlled autonomous driving system comprising:

one or more controllers executing instructions to:

receive an electrical signal representative of the voice command generated by an occupant of the vehicle, wherein the voice command indicates a driving maneuver that alters motion of the vehicle;

convert the electrical signal into a text-based command based on one or more speech recognition algorithms;

classify the text-based command into one of a plurality of pre-compiled driving maneuvers based on one or more machine learning algorithms, wherein the pre-compiled driving maneuvers represent potential driving maneuvers that the vehicle has the potential to execute, and wherein the pre-compiled driving maneuvers include at least the following: take the next exit, to make a lane change, and to pass another vehicle;

classify the pre-compiled driving maneuver as either a route-based driving maneuver or a behavior-based driving maneuver;

in response to classifying the pre-compiled driving maneuver as the route-based driving maneuver, compare the pre-compiled driving maneuver with map data to identify one or more relevant lane segments of the map data affected by the pre-compiled driving maneuver;

determine an updated route plan by updating a current route plan the vehicle is presently following to include the one or more relevant lane segments, wherein the trajectory of the vehicle is determined based on the updated route plan;

in response to classifying the pre-compiled driving maneuver as a behavior-based driving maneuver, compare the pre-compiled driving maneuver with a plurality of behavior planner parameters to determine one or more updated behavior planner parameters;

determine an updated goal state of the vehicle based on the one or more updated behavior planner parameters; and

instruct the vehicle to perform at least one of accelerating, braking, and steering to follow the trajectory.

16 . The voice controlled autonomous driving system of claim 15 , wherein the updated goal state indicates a driving maneuver, the one or more updated behavior planner parameters, and the behavior planner parameters that are followed to execute the pre-compiled driving maneuver and the trajectory of the vehicle is determined based on the updated goal state.

17 . The voice controlled autonomous driving system of claim 15 , wherein a confidence score indicating a confidence of the one or more machine learning algorithms that classify the text-based command is associated with the pre-compiled driving maneuver.

18 . The voice controlled autonomous driving system of claim 17 , further comprising a human-machine interface (HMI) in electronic communication with the one or more controllers.

19 . The voice controlled autonomous driving system of claim 18 , wherein the one or more controllers execute instructions to:

compare the confidence score associated with the pre-compiled driving maneuver with a confidence threshold value; and

in response to determining the confidence score associated with the pre-compiled driving maneuver is less than the confidence threshold value, instruct the HMI to generate a message indicating the voice controlled autonomous driving system is unable to execute the voice command.

20 . The voice controlled autonomous driving system of claim 15 , further comprising a microphone in electronic communication with the one or more controllers, wherein the microphone converts the voice command into the electrical signal representative of the voice command.