IP Library › Granted Patent US 12,077,160
Granted Patent B2
US 12,077,160 · App. 17/090,392 · Granted Sep 3, 2024

Apparatus and method for performing an action associated with a driver input indicated via an input modality of a human machine interface of a vehicle

Inventors: Hiroshi Yokoyama (Tokyo, JP); Ercan Mehmet Dede (Ann Arbor, MI)
Assignee: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
B60W40/08B60K35/00B60W30/0956G06V20/58G06V20/597B60K35/10B60K35/20B60K2360/143B60W2420/403B60W2420/408B60W2420/54B60W2540/229G01C21/3605G06V2201/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,077,160
App. No.
17/090,392
Granted
Sep 3, 2024
Kind
B2
Abstract

The present disclosure relates to driver interaction with a user interface of a vehicle. In particular, the present disclosure relates to an apparatus for performing an action associated with a driver input indicated via an input modality of an HMI of a vehicle, the apparatus comprising processing circuitry configured to receive initial data corresponding to an initial driver input via a first input modality of the HMI of the vehicle, compare a confidence level of the initial data to a confidence level threshold associated with the first input modality of the HMI of the vehicle, request, based on the comparison, a subsequent driver input via a second input modality of the HMI of the vehicle, determine a presence of an adverse condition within a vehicle environment, and perform an action associated with the subsequent driver input based on the presence of the adverse condition within the vehicle environment.

Claims (50)

1. An apparatus for performing an action associated with a driver input indicated via an input modality of a human machine interface of a vehicle, the apparatus comprising:

processing circuitry configured to

receive initial data corresponding to an initial driver input via a first input modality of the human machine interface of the vehicle,

compare a confidence level of the initial data to a confidence level threshold associated with the first input modality of the human machine interface of the vehicle,

request, based on the comparison, a subsequent driver input via a second input modality of the human machine interface of the vehicle,

receive the subsequent driver input via the second input modality,

determine a presence of an adverse condition within a vehicle environment after receiving the subsequent driver input,

delay performing an action associated with the subsequent driver input when the adverse condition is present within the vehicle environment by entering, into a buffer, the action associated with the subsequent driver input, and determining, iteratively, whether the adverse condition is present within the vehicle environment, wherein the action is an adjustment of one of a plurality of systems of the vehicle, and

perform the action associated with the subsequent driver input when it is determined that the adverse condition is not present within the vehicle environment.

2. The apparatus according to claim 1 , wherein the processing circuitry is configured to determine the presence of the adverse condition within the vehicle environment by

detecting, via external sensors of the vehicle, a presence of an emergency vehicle within the vehicle environment.

3. The apparatus according to claim 2 , wherein the processing circuitry is configured to detect the presence of the emergency vehicle by audio recognition.

4. The apparatus according to claim 2 , wherein the processing circuitry is configured to detect the presence of the emergency vehicle by image recognition.

5. The apparatus according to claim 1 , wherein the processing circuitry is configured to determine the presence of the adverse condition within the vehicle environment by

detecting, via internal sensors of the vehicle, a presence of a distracting event within the vehicle environment, the distracting event being inside a cabin of the vehicle.

6. The apparatus according to claim 1 , wherein the processing circuitry is configured to request the subsequent driver input via the second input modality of the human machine interface of the vehicle by

receiving data from each of a plurality of input modalities of the human machine interface of the vehicle,

comparing a confidence level of the data from each of the plurality of input modalities of the human machine interface of the vehicle to a respective confidence level threshold associated with each of the plurality of input modalities of the human machine interface of the vehicle,

ranking the plurality of input modalities of the human machine interface of the vehicle, based on the comparing, from lowest confidence to highest confidence, and

selecting, as the second input modality of the human machine interface of the vehicle, an input modality of the plurality of input modalities of the human machine interface of the vehicle having the highest confidence.

7. The apparatus according to claim 6 , wherein the plurality of input modalities of the human machine interface of the vehicle include infrared imaging units, cameras, touch screen interfaces, ultrasonic imaging units, haptic interfaces, steering wheel tactile pads, radars, or a combination thereof.

8. A method for performing an action associated with a driver input indicated via an input modality of a human machine interface of a vehicle, the method comprising:

receiving initial data corresponding to an initial driver input via a first input modality of the human machine interface of the vehicle;

comparing a confidence level of the initial data to a confidence level threshold associated with the first input modality of the human machine interface of the vehicle;

requesting, based on the comparison, a subsequent driver input via a second input modality of the human machine interface of the vehicle;

receiving the subsequent driver input via the second input modality;

determining a presence of an adverse condition within a vehicle environment after receiving the subsequent driver input;

delaying performing an action associated with the subsequent driver input when the adverse condition is present within the vehicle environment by entering, into a buffer, the action associated with the subsequent driver input, and determining, iteratively, whether the adverse condition is present within the vehicle environment, wherein the action is an adjustment of one of a plurality of systems of the vehicle; and

performing the action associated with the subsequent driver input when it is determined that the adverse condition is not present within the vehicle environment.

9. The method according to claim 8 , wherein the determining the presence of the adverse condition within the vehicle environment includes

detecting, via external sensors of the vehicle, a presence of an emergency vehicle within the vehicle environment.

10. The method according to claim 9 , further comprising detecting the presence of the emergency vehicle by audio recognition or by image recognition.

11. The method according to claim 8 , wherein the determining the presence of the adverse condition within the vehicle environment includes

detecting, via internal sensors of the vehicle, a presence of a distracting event within the vehicle environment, the distracting event being inside a cabin of the vehicle.

12. A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method of performing an action associated with a driver input indicated via an input modality of a human machine interface of a vehicle, comprising:

receiving initial data corresponding to an initial driver input via a first input modality of the human machine interface of the vehicle;

comparing a confidence level of the initial data to a confidence level threshold associated with the first input modality of the human machine interface of the vehicle;

requesting, based on the comparison, a subsequent driver input via a second input modality of the human machine interface of the vehicle;

receiving the subsequent driver input via the second input modality;

determining a presence of an adverse condition within a vehicle environment after receiving the subsequent driver input;

delaying performing an action associated with the subsequent driver input when the adverse condition is present within the vehicle environment by entering, into a buffer, the action associated with the subsequent driver input, and determining, iteratively, whether the adverse condition is present within the vehicle environment, wherein the action is an adjustment of one of a plurality of systems of the vehicle; and

performing the action associated with the subsequent driver input when it is determined that the adverse condition is not present within the vehicle environment.

13. The non-transitory computer-readable storage medium according to claim 12 , wherein the determining the presence of the adverse condition within the vehicle environment includes

detecting, via external sensors of the vehicle, a presence of an emergency vehicle within the vehicle environment.

14. The non-transitory computer-readable storage medium according to claim 12 , wherein the requesting the subsequent driver input via the second input modality of the human machine interface of the vehicle includes

receiving data from each of a plurality of input modalities of the human machine interface of the vehicle,

comparing a confidence level of the data from each of the plurality of input modalities of the human machine interface of the vehicle to a respective confidence level threshold associated with each of the plurality of input modalities of the human machine interface of the vehicle,

ranking the plurality of input modalities of the human machine interface of the vehicle, based on the comparing, from lowest confidence to highest confidence, and

selecting, as the second input modality of the human machine interface of the vehicle, an input modality of the plurality of input modalities of the human machine interface of the vehicle having the highest confidence.

15. The apparatus according to claim 1 , wherein the action is one of an adjustment of an audio system of the vehicle, an adjustment of a lighting system of the vehicle, or an adjustment of a navigation system of the vehicle.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2024
From: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 068894/0442 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 5, 2020
From: YOKOYAMA, HIROSHI; DEDE, ERCAN MEHMET
To: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
Reel/Frame 054288/0773 →
Continuity (1)
Related Publication 20220135048A1 · May 5, 2022