IP Library Granted Patent US 11,396,271
Granted Patent B2
US 11,396,271 · App. 16/905,348 · Granted Jul 26, 2022

System and method for communicating between autonomous vehicle and vulnerable road users

Inventor: Helen Elizabeth Kourous-Harrigan (Monroe, MI)
Assignee: Ford Global Technologies, LLC
B60R21/34B60R21/0132B60W60/0017G06T19/006H04W4/44B60W2554/4029B60W2554/4047
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,396,271
App. No.
16/905,348
Granted
Jul 26, 2022
Kind
B2
Abstract

The present disclosure relates to a method and system for communication between a vulnerable road user and an autonomous vehicle using augmented reality to highlight information to the vulnerable road user regarding potential interactions between the autonomous vehicle and the vulnerable road user.

Claims (52)

1. A computer-implemented method for communicating between a mobile device of a vulnerable road user (VRU) and an autonomous vehicle (AV), the method comprising:

determining, via a processor of the mobile device, that the AV is proximate the mobile device of the VRU;

receiving, by the processor of the mobile device from the AV, navigational intent comprising a predicted path for the AV on a roadway;

generating, by the processor of the mobile device, an impact event prediction indicative of a future impact event between the AV and the mobile device of the VRU, wherein the impact event prediction is based on the predicted path for the AV; and

displaying, via a mobile device display, a camera image of the roadway and an augmented reality (AR) overlay, wherein the AR overlay depicts a representation of the future impact event.

2. The method according to claim 1 , wherein generating the impact event prediction comprises:

determining, using a threshold value for comparison, that the predicted path for the AV is likely to result in the future impact event between the AV and the VRU; and

determining a time to collision value using the predicted path for the AV and a location of the mobile device, responsive to determining that the predicted path for the AV is likely to result in the future impact event given the proximity of the VRU.

3. The method according to claim 1 , further comprising:

generating a value indicative of a severity of the future impact event; and

displaying the AR overlay on the mobile device of the VRU through the AR overlay, wherein displaying comprises animating a graphic associated with the severity of the future impact event.

4. The method according to claim 3 , further comprising:

determining, via the processor of the mobile device, a recommendation; and

generating the recommendation via the AR overlay.

5. The method according to claim 4 , further comprising:

transmitting, to the AV, a second message comprising a path alteration instruction for altering the predicted path of the AV, wherein the instruction is configured to cause the AV to mitigate or avoid the future impact event.

6. The method according to claim 5 , wherein the path alteration instruction comprises an instruction for altering at least one of a vehicle speed and a vehicle direction of the AV.

7. The method according to claim 1 , further comprising:

receiving map data from a map provider database at the mobile device; and

displaying the AR overlay on the mobile device through the AR overlay, wherein the AR overlay is based on the map provider data.

8. The method according to claim 7 , wherein the map provider database is associated with an AV fleet comprising a fleet of autonomous vehicles.

9. The method according to claim 7 , wherein the map provider database is associated with a municipal map provider.

10. The method according to claim 7 , further comprising:

augmenting, via the processor of the mobile device, the map data; and

displaying augmented map data on the mobile device of the VRU through the AR overlay.

11. The method according to claim 10 , wherein augmenting the map data comprises:

generating a graphic on the mobile device depicting the map, a representation of the predicted path for the AV, and the AR overlay depicting the future impact event.

12. A system for communicating between a mobile device of a vulnerable road user (VRU) and an autonomous vehicle (AV), the system comprising:

a processor; and

a memory for storing computer-executable instructions, the processor programmed to execute the instructions to:

determine that an autonomous vehicle (AV) on a roadway is proximate the mobile device;

determine a navigational intent of the AV comprising a predicted path for the AV;

generate an impact event prediction indicative of a potential future impact event between the AV and the mobile device, wherein the impact event prediction is based on the predicted path for the AV; and

display, via a display, a camera image of the roadway and an augmented reality (AR) overlay, wherein the AR overlay depicts a representation of the potential future impact event.

13. The system of claim 12 , wherein the processor is programmed to generate the impact event prediction by executing an instruction to:

determine, using a threshold value for comparison, that the predicted path for the AV is likely to result in the future impact event; and

determine a time to collision value using the predicted path for the AV and a location of the mobile device, responsive to determining that the predicted path for the AV is likely to result in the future impact event.

14. The system of claim 12 , wherein the processor is programmed to generate the impact event prediction by executing an instruction to:

generate a value indicative of a severity of the future impact event; and

display map on the mobile device through the AR overlay, wherein displaying comprises animating a graphic associated with the severity of the potential future impact event.

15. The system of claim 12 , wherein the processor is programmed to execute an instruction to:

determine a recommendation; and

generate the recommendation via the AR overlay.

16. The system of claim 15 , wherein the processor is programmed to execute an instruction to:

determine a path alteration instruction for altering the predicted path of the AV, wherein the instruction is programmed to cause the AV to mitigate or avoid the potential future impact event.

17. The system of claim 16 , wherein the path alteration instruction comprises an instruction for altering at least one of a vehicle speed and a vehicle direction of the AV.

18. The system of claim 12 , wherein the processor is programmed to execute an instruction to:

receive map data from a map provider database; and

display the map on the mobile device containing the AR overlay, wherein the map is based on the map provider data.

19. The system of claim 18 , wherein the processor is programmed to execute the instruction to:

augment, via the processor of the mobile device, the map data; and

display augmented map data on the mobile device through the AR overlay.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2021
From: KOUROUS-HARRIGAN, HELEN ELIZABETH
To: FORD GLOBAL TECHNOLOGIES, LLC
Reel/Frame 054813/0167 →
Continuity (1)
Related Publication 20210394705A1 · Dec 23, 2021
Cited By (1)
US 12,725,523