IP Library Granted Patent US 10,328,949
Granted Patent B2
US 10,328,949 · App. 15/008,518 · Granted Jun 25, 2019

Sensor blind spot indication for vehicles

Inventor: Nobuhide Kamata (Ann Arbor, MI)
Assignee: Toyota Motor Engineering & Manufacturing North America, Inc.
B60W50/085B60K35/00B60W50/14G07C5/0825G07C5/0833G08G1/166G08G1/167B60K2350/2052B60W2050/143H04L67/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,328,949
App. No.
15/008,518
Granted
Jun 25, 2019
Kind
B2
Abstract

A vehicle can be configured to indicate sensor blind spots to a vehicle occupant (e.g., a driver). Using one or more sensors, the vehicle can acquire driving environment data of an external environment of the vehicle. It can be determined whether one or more portions of the acquired driving environment data is unreliable. Responsive to determining that one or more portions of the acquired driving environment data is unreliable, an alert can be caused to be presented within the vehicle. The alert can indicate one or more locations in the external environment in which the driving environment data that is determined to be unreliable. In some instances, the alert can be a visual alert and/or an audial alert.

Claims (101)

1. A method of indicating sensor blind spots for an autonomous vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including:

determining whether one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors; and

responsive to determining that one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by the at least one of the one or more sensors, determining that the portion of the acquired driving environment data associated with the at least one of the one or more sensors that are occluded by one or more physical objects to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

2. The method of claim 1 , wherein the alert is a visual alert, and wherein causing the visual alert to be presented within the autonomous vehicle includes causing the visual alert to be presented on a display within the autonomous vehicle.

3. The method of claim 2 , wherein the visual alert includes a graphical representation of the autonomous vehicle and a graphical representation of the one or more portions of the external environment is not reliably detectable by the one or more sensors.

4. The method of claim 1 , wherein the alert is a wherein the alert is an audial alert.

5. The method of claim 4 , wherein causing the audial alert to be presented within the autonomous vehicle includes causing the audial alert to be presented within the autonomous vehicle in one or more locations that spatially correspond to the one or more locations in the external environment in which the acquired driving environment data that is determined to be unreliable.

6. A method of indicating sensor blind spots for an autonomous vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including determining whether sensor data is completely occluded in one or more portions of the external environment of the autonomous vehicle;

responsive to determining that sensor data is completely occluded in one or more portions of the external environment, causing the autonomous vehicle to transition to a manual operational mode or safety operational mode; and

causing an alert to be presented within the autonomous vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

7. The method of claim 1 , further including:

responsive to determining that one or more portions of the acquired driving environment data is unreliable, determining a modification to a current driving maneuver of the autonomous vehicle; and

causing the determined modification to the current driving maneuver of the autonomous vehicle to be implemented.

8. The method of claim 7 , wherein the determined modification to a current driving maneuver of the autonomous vehicle includes reducing a current speed of the autonomous vehicle.

9. The method of claim 1 , wherein determining whether one or more portions of the acquired driving environment data is unreliable includes:

detecting whether any of the one or more sensors are inoperative or faulting; and

responsive to detecting that any of the one or more sensors are inoperative or faulting, determining that the portion of the acquired driving environment data associated with the inoperative or faulting sensors to be unreliable.

10. A method of indicating sensor blind spots for a vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including:

comparing the acquired driving environment data to a map;

determining whether there is a difference between the acquired driving environment data and an expected feature included in the map; and

responsive to determining whether there is a difference between the acquired driving environment data and an expected feature included in the map, determining that that portion of the acquired driving environment data associated with the difference between the acquired driving environment data and an expected feature included in the map to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

11. A sensor blind spot indication system for an autonomous vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the autonomous vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising:

determining whether one or more portions of acquired driving environment data is unreliable, the determining including:

determining whether one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors; and

responsive to determining that one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors, determining that the portion of the acquired driving environment data associated with the one or more sensors that are occluded by at least one of one or more physical objects to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

12. The system of claim 11 , further including a display, wherein the display is operatively connected to the processor, wherein the alert is a visual alert, and wherein causing an alert to be presented within the autonomous vehicle includes causing the visual alert to be presented on the display.

13. The system of claim 11 , wherein the alert is a visual alert, and further including:

a projector; and

a projection surface,

wherein causing an alert to be presented within the autonomous vehicle includes causing the projector to present the visual alert on the projection surface.

14. The system of claim 13 , wherein the visual alert includes a graphical representation of the autonomous vehicle and a graphical representation of the one or more portions of the external environment is not reliably detectable by the one or more sensors.

15. The system of claim 11 , further including a plurality of speakers, wherein the plurality of speakers are operatively connected to the processor, wherein the alert is a audial alert, and wherein causing an alert to be presented within the autonomous vehicle includes causing the audial alert to be presented by one or more of the plurality of speakers.

16. The system of claim 15 , wherein causing the audial alert to be presented within the autonomous vehicle includes causing the audial alert to be presented within the autonomous vehicle by one or more of the plurality of speakers in one or more locations that spatially correspond to the one or more locations in the external environment in which the acquired driving environment data that is determined to be unreliable.

17. The system of claim 11 , wherein the executable operations further include:

responsive to determining that one or more portions of the acquired driving environment data is unreliable, determining a modification to a current driving maneuver of the autonomous vehicle; and

causing the determined modification to the current driving maneuver of the autonomous vehicle to be implemented.

18. The system of claim 17 , wherein the determined modification to a current driving maneuver of the autonomous vehicle includes reducing a current speed of the autonomous vehicle.

19. A computer program product for indicating sensor blind spots for an autonomous vehicle, the computer program product comprising a computer readable storage medium having program code embodied therein, the program code executable by a processor to perform a method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including:

determining whether one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors; and

responsive to determining that one or more physical objects or barriers in the external environment occlude portions of the external environment from being detected by at least one of the one or more sensors, determining that the portion of the acquired driving environment data associated with the at least one of the one or more sensors that are occluded by one or more physical objects to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

20. A sensor blind spot indication system for a vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising:

determining whether one or more portions of acquired driving environment data is unreliable, the determining including:

comparing the acquired driving environment data to a map;

determining whether there is a difference between the acquired driving environment data and an expected feature included in the map; and

responsive to determining whether there is a difference between the acquired driving environment data and an expected feature included in the map, determining that that portion of the acquired driving environment data associated with the difference between the acquired driving environment data and an expected feature included in the map to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

21. A computer program product for indicating sensor blind spots for a vehicle, the computer program product comprising a computer readable storage medium having program code embodied therein, the program code executable by a processor to perform a method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including:

comparing the acquired driving environment data to a map;

determining whether there is a difference between the acquired driving environment data and an expected feature included in the map; and

responsive to determining whether there is a difference between the acquired driving environment data and an expected feature included in the map, determining that that portion of the acquired driving environment data associated with the difference between the acquired driving environment data and an expected feature included in the map to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

22. A method of indicating sensor blind spots for an autonomous vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of the acquired driving environment data is unreliable, the determining including:

determining whether map data is available for a current external environment of the autonomous vehicle; and

responsive to determining that map data is not available for at least a portion of the current external environment of the autonomous vehicle, determining that the acquired driving environment data associated with the at least a portion of the current external environment of the autonomous vehicle for which map data is not available to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

23. A sensor blind spot indication system for an autonomous vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the autonomous vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising:

determining whether one or more portions of acquired driving environment data is unreliable, the determining including:

determining whether map data is available for a current external environment of the autonomous vehicle; and

responsive to determining that map data is not available for at least a portion of the current external environment of the autonomous vehicle, determining that the acquired driving environment data associated with the at least a portion of the current external environment of the autonomous vehicle for which map data is not available to be unreliable; and

responsive to determining that one or more portions of the acquired driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

24. A method of indicating sensor blind spots for an autonomous vehicle, the method comprising:

acquiring, using one or more sensors, driving environment data of an external environment of the autonomous vehicle;

determining whether one or more portions of driving environment data is unreliable, the determining including:

determining whether one or more environment sensors is off or deactivated;

responsive to determining that one or more environment sensors is off or deactivated, determining that the portion of the driving environment data associated with the one or more environment sensors that are off or deactivated to be unreliable; and

responsive to determining that one or more portions of the driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the driving environment data that is determined to be unreliable.

25. A sensor blind spot indication system for an autonomous vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the autonomous vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising:

determining whether one or more portions of driving environment data is unreliable, the determining including:

determining whether one or more environment sensors is off or deactivated; and

responsive to determining that one or more environment sensors is off or deactivated, determining that the portion of the driving environment data associated with the one or more environment sensors that are off or deactivated to be unreliable; and

responsive to determining that one or more portions of the driving environment data is unreliable, causing an alert to be presented within the autonomous vehicle while maintaining an autonomous operational mode, the alert indicating one or more locations in the external environment which correspond to the driving environment data that is determined to be unreliable.

26. A sensor blind spot indication system for an autonomous vehicle, the system comprising:

one or more sensors configured to acquire driving environment data of at least a portion of an external environment of the autonomous vehicle; and

a processor operatively connected to the one or more sensors, the processor being programmed to initiate executable operations comprising:

determining whether one or more portions of acquired driving environment data is unreliable, the determining including determining whether sensor data is completely occluded in one or more portions of an external environment of the autonomous vehicle;

responsive to determining that sensor data is completely occluded in one or more portions of the external environment, causing the autonomous vehicle to transition to a manual operational mode or safety operational mode; and

causing an alert to be presented within the autonomous vehicle, the alert indicating one or more locations in the external environment which correspond to the acquired driving environment data that is determined to be unreliable.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2019
From: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 051201/0710 →
CHANGE OF ADDRESS Recorded Nov 30, 2018
From: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
To: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
Reel/Frame 047688/0784 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2016
From: KAMATA, NOBUHIDE
To: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
Reel/Frame 037805/0388 →
Continuity (1)
Related Publication 20170221359A1 · Aug 3, 2017