Virtualized driver assistance

1. A method comprising:

determining a reference vehicle traveling in a first lane, the first lane merging with a second lane at a merging point;

determining a merging plan for a merging vehicle traveling in the second lane to merge into the first lane based on a vehicle position of the reference vehicle in the first lane; and

overlaying, via a display device, a virtual target representing the reference vehicle in a field of view of a roadway environment of a driver of the merging vehicle based on the merging plan of the merging vehicle including displaying a projected location of the reference vehicle ahead of the merging vehicle in the field of view, a size of the virtual target on the display device being inversely proportional to a distance of the projected location of the reference vehicle from the merging vehicle.

2. The method of claim 1 , wherein the display device comprises one of a head-up display unit, smart glasses, and a headset.

3. The method of claim 1 , wherein

determining the merging plan includes determining a simulated position for the reference vehicle in the second lane ahead of the merging vehicle in the second lane based on the vehicle position of the reference vehicle in the first lane; and

overlaying the virtual target in the field of view includes rendering the virtual target in the field of view at a display position that is based on the simulated position of the reference vehicle.

4. The method of claim 3 , wherein overlaying the virtual target in the field of view includes:

determining the display position of the virtual target in the field of view based on the simulated position of the reference vehicle and one or more roadway attributes of the second lane; and

rendering the virtual target at the display position in the field of view.

5. The method of claim 3 , further comprising:

determining a position range for the merging vehicle in the second lane based on the simulated position of the reference vehicle in the second lane; and

overlaying a virtual position indicator indicating the position range for the merging vehicle in the field of view.

6. The method of claim 3 , wherein determining the simulated position of the reference vehicle in the second lane includes:

receiving vehicle movement data of the reference vehicle in the first lane;

receiving vehicle movement data of the merging vehicle in the second lane;

receiving roadway data describing the first lane and second lane; and

determining the simulated position of the reference vehicle in the second lane based on the vehicle movement data of the reference vehicle in the first lane, the vehicle movement data of the merging vehicle in the second lane, and the roadway data.

7. The method of claim 3 , further comprising:

receiving driving behavior data associated with a driver of the reference vehicle;

receiving sensor data captured by one or more sensors of the merging vehicle; and

adjusting the simulated position of the reference vehicle in the second lane based on the driving behavior data associated with the driver of the reference vehicle and the sensor data of the merging vehicle.

8. The method of claim 1 , wherein

the field of view of the roadway environment includes a front field of view and a side field of view associated with the driver of the merging vehicle;

overlaying the virtual target in the field of view includes rendering the virtual target on a front display surface, the front display surface covering at least a portion of a windshield of the merging vehicle that is within the front field of view; and the method includes:

overlaying a safety information indicator in the field of view, the safety information indicator being rendered on a side display surface, the side display surface covering at least a portion of a side window or a portion of a side mirror of the merging vehicle that is within the side field of view.

9. The method of claim 1 , wherein determining the reference vehicle traveling in the first lane includes:

estimating one or more merging timestamps at which one or more vehicles traveling in the first lane and the second lane arrive at the merging point, the one or more estimated merging timestamps including an estimated merging timestamp of the merging vehicle; and

determining the reference vehicle from the one or more vehicles based on the estimated merging timestamp of the merging vehicle.

10. The method of claim 9 , wherein determining the reference vehicle from the one or more vehicles includes:

determining a first merging timestamp that has a lowest time distance prior to the estimated merging timestamp of the merging vehicle among the one or more estimated merging timestamps; and

determining the reference vehicle based on the first merging timestamp.

11. The method of claim 9 , further comprising:

monitoring the one or more vehicles traveling in the first lane and the second lane;

updating the one or more estimated merging timestamps at which the one or more vehicles arrive at the merging point;

determining a different reference vehicle traveling in the first lane based on the one or more updated estimated merging timestamps of the one or more vehicles;

responsive to determining the different reference vehicle traveling in the first lane, updating the merging plan of the merging vehicle based on a vehicle position of the different reference vehicle in the first lane; and

adjusting the virtual target overlaid in the field of view based on the updated merging plan of the merging vehicle.

12. The method of claim 1 , wherein the first lane is on a freeway and the second lane is on a ramp segment of the freeway.

13. A system comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the system to:

determine a reference vehicle traveling in a first lane, the first lane merging with a second lane at a merging point;

determine a merging plan for a merging vehicle traveling in the second lane to merge into the first lane based on a vehicle position of the reference vehicle in the first lane; and

overlay, via a display device, a virtual target in a field of view of a roadway environment of a driver of the merging vehicle based on the merging plan of the merging vehicle including displaying a projected location of the reference vehicle ahead of the merging vehicle in the field of view, a size of the virtual target on the display device being inversely proportional to a distance of the projected location of the reference vehicle from the merging vehicle.

14. The system of claim 13 , wherein the display device comprises one of a head-up display unit, smart glasses, and a headset.

15. The system of claim 13 , wherein

to determine the merging plan includes determining a simulated position for the reference vehicle in the second lane ahead of the merging vehicle in the second lane based on the vehicle position of the reference vehicle in the first lane; and

to overlay the virtual target in the field of view includes rendering the virtual target in the field of view at a display position that is based on the simulated position of the reference vehicle.

16. The system of claim 15 , wherein to overlay the virtual target in the field of view includes:

determining the display position of the virtual target in the field of view based on the simulated position of the reference vehicle and one or more roadway attributes of the second lane; and

rendering the virtual target at the display position in the field of view.

17. The system of claim 15 , wherein the instructions, when executed by the one or more processors, further cause the system to:

determine a position range for the merging vehicle in the second lane based on the simulated position of the reference vehicle in the second lane; and

overlay a virtual position indicator indicating the position range for the merging vehicle in the field of view.

18. The system of claim 15 , wherein to determine the simulated position of the reference vehicle in the second lane includes:

receiving vehicle movement data of the reference vehicle in the first lane;

receiving vehicle movement data of the merging vehicle in the second lane;

receiving roadway data describing the first lane and second lane; and

determining the simulated position of the reference vehicle in the second lane based on the vehicle movement data of the reference vehicle in the first lane, the vehicle movement data of the merging vehicle in the second lane, and the roadway data.

19. The system of claim 15 , wherein the instructions, when executed by the one or more processors, further cause the system to:

receive driving behavior data associated with a driver of the reference vehicle;

receive sensor data captured by one or more sensors of the merging vehicle; and

adjust the simulated position of the reference vehicle in the second lane based on the driving behavior data associated with the driver of the reference vehicle and the sensor data of the merging vehicle.

20. The system of claim 13 , wherein

the field of view of the roadway environment includes a front field of view and a side field of view associated with the driver of the merging vehicle;

to overlay the virtual target in the field of view includes rendering the virtual target on a front display surface, the front display surface covering at least a portion of a windshield of the merging vehicle that is within the front field of view; and

the instructions, when executed by the one or more processors, further cause the system to overlay a safety information indicator in the field of view, the safety information indicator being rendered on a side display surface, the side display surface covering at least a portion of a side window or a portion of a side mirror of the merging vehicle that is within the side field of view.

21. The system of claim 13 , wherein to determine the reference vehicle traveling in the first lane includes:

estimating one or more merging timestamps at which one or more vehicles traveling in the first lane and the second lane arrive at the merging point, the one or more estimated merging timestamps including an estimated merging timestamp of the merging vehicle; and

determining the reference vehicle from the one or more vehicles based on the estimated merging timestamp of the merging vehicle.

22. The system of claim 21 , wherein to determine the reference vehicle from the one or more vehicles includes:

determining a first merging timestamp that has a lowest time distance prior to the estimated merging timestamp of the merging vehicle among the one or more estimated merging timestamps; and

determining the reference vehicle based on the first merging timestamp.

23. The system of claim 21 , wherein the instructions, when executed by the one or more processors, further cause the system to:

monitor the one or more vehicles traveling in the first lane and the second lane;

update the one or more estimated merging timestamps at which the one or more vehicles arrive at the merging point;

determine a different reference vehicle traveling in the first lane based on the one or more updated estimated merging timestamps of the one or more vehicles;

responsive to determining the different reference vehicle traveling in the first lane, update the merging plan of the merging vehicle based on a vehicle position of the different reference vehicle in the first lane; and

adjust the virtual target overlaid in the field of view based on the updated merging plan of the merging vehicle.

24. The system of claim 13 , wherein the first lane is on a freeway and the second lane is on a ramp segment of the freeway.

25. A method comprising:

determining a reference vehicle traveling in a first lane, the first lane merging with a second lane at a merging point;

determining a merging plan for a first merging vehicle traveling in the second lane to merge into the first lane based on a vehicle position of the reference vehicle in the first lane;

overlaying, via a first display device, a first virtual target in a field of view of a driver of the first merging vehicle in a roadway environment based on the merging plan of the first merging vehicle including displaying a projected location of the reference vehicle in the second lane ahead of the first merging vehicle in the field of view of the driver of the first merging vehicle, a size of the virtual target on the display device being inversely proportional to a distance of the projected location of the reference vehicle from the merging vehicle;

determining a second merging vehicle traveling in the second lane;

determining a merging plan for the reference vehicle to merge with the second merging vehicle as the second merging vehicle merges into the first lane based on a vehicle position of the second merging vehicle in the second lane; and

overlaying, via a second display device, a second virtual target in a field of view of the roadway environment of a driver of the reference vehicle based on the merging plan of the reference vehicle.

26. The method of claim 25 , further comprising:

estimating one or more merging timestamps at which one or more vehicles traveling in the first lane and the second lane arrive at the merging point;

determining the reference vehicle from the one or more vehicles based on an estimated merging timestamp of the first merging vehicle among the one or more estimated merging timestamps; and

determining the second merging vehicle from the one or more vehicles based on an estimated merging timestamp of the reference vehicle among the one or more estimated merging timestamps.