IP Library Granted Patent US 12675109
Granted Patent B2
US 12675109 · App. 18/194,333 · Granted Jul 7, 2026

Displaying an overlay including a projected path of a vehicle

Inventors: Liam Pedersen (San Francisco, CA); Viju James (Fremont, CA); Mark Fischer (Santa Rosa, CA); D M Raisul Ahsan (San Francisco, CA)
Assignee: Nissan North America, Inc.
G05D1/0038G05D1/0016G06V10/751G06V20/588
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12675109
App. No.
18/194,333
Granted
Jul 7, 2026
Kind
B2
Abstract

A system may receive a video stream from a camera of a vehicle in a transportation network. The system may also receive an input indicating a steering angle and/or a distance to travel. The system may determine a projected path of the vehicle in the transportation network based on the input. The projected path may be determined using spatial coordinates relative to the transportation network or the camera. The system may then determine, from the spatial coordinates, pixel coordinates in the video stream corresponding to the projected path. The system may display an overlay in the video stream based on the pixel coordinates. The overlay may include a visualization of the projected path.

Claims (48)

1 . A method, comprising:

receiving, at a remote system, a video stream from a camera of a vehicle in a transportation network, wherein the remote system is external from the vehicle;

receiving, at the remote system, an input indicating at least one of a steering angle or a distance to travel;

determining, at the remote system, a projected path of the vehicle in the transportation network based on the input, wherein the projected path is determined using spatial coordinates relative to at least one of the transportation network or the camera;

determining, at the remote system from the spatial coordinates and camera parameters associated with the camera, pixel coordinates in the video stream corresponding to the projected path;

determining, at the remote system, a predicted position of the vehicle in the video stream based on a latency associated with the video stream; and

displaying, at the remote system, an overlay in the video stream based on the pixel coordinates, the overlay including a visualization of the projected path starting from the predicted position.

2 . The method of claim 1 , further comprising:

receiving, at the remote system, a selection of the overlay; and

transmitting, by the remote system, information to the vehicle based on the selection, the information configured to cause the vehicle to follow the projected path.

3 . The method of claim 1 ,

wherein the camera parameters comprise an orientation of the camera and a resolution of the camera.

4 . The method of claim 1 ,

wherein the overlay further includes an indication of the latency, and wherein the indication is color coded with a first color for lesser latency, a second color for moderate latency, or a third color for greater latency.

5 . The method of claim 1 , wherein the overlay is represented by lines corresponding to a lane of travel.

6 . The method of claim 1 , wherein the input is received through a graphical user interface (GUI) by a detecting a mouse hovering over an area in the GUI to configure the steering angle, and wherein a selection of the overlay is received by detecting clicking of a button of the mouse.

7 . The method of claim 1 , wherein determining the projected path includes applying Ackermann steering geometry to determine a radius of the projected path.

8 . The method of claim 1 , wherein displaying the overlay includes applying spline interpolation based on the pixel coordinates.

9 . An apparatus, comprising:

a memory; and

a processor located at a remote system configured to execute instructions stored in the memory to:

receive a video stream from a camera of a vehicle in a transportation network, wherein the remote system is external from the vehicle;

receive an input indicating a steering angle and a distance to travel;

determine a projected path of the vehicle in the transportation network based on the steering angle and the distance to travel, wherein the projected path is determined using three dimensional spatial coordinates relative to the transportation network;

determine, from the three dimensional spatial coordinates and camera parameters associated with the camera, two dimensional pixel coordinates in the video stream corresponding to the projected path;

determine a predicted position of the vehicle in the video stream based on a latency associated with the video stream; and

display an overlay in the video stream based on the two dimensional pixel coordinates, the overlay including a visualization of the projected path starting from the predicted position.

10 . The apparatus of claim 9 , wherein the processor is further configured to execute instructions stored in the memory to:

receive a selection of the overlay; and

transmit information to the vehicle based on the selection, the information configured to cause the vehicle to follow the projected path.

11 . The apparatus of claim 9 , wherein the camera parameters comprise an orientation of the camera and a resolution of the camera.

12 . The apparatus of claim 9 , wherein the overlay further includes an indication of the latency, and wherein the indication is color coded with a first color for lesser latency, a second color for moderate latency, or a third color for greater latency.

13 . The apparatus of claim 9 , wherein the overlay is represented by lines corresponding to a lane of travel.

14 . The apparatus of claim 9 , wherein the input is received through a graphical user interface (GUI) by a detecting a mouse hovering over an area in the GUI to configure the steering angle and by detecting a wheel of the mouse that is scrolling to configure the distance, and wherein a selection of the overlay is received by detecting clicking of a button of the mouse.

15 . A non-transitory computer readable medium storing instructions operable to cause one or more processors to perform operations comprising:

receiving, at a remote system, a video stream from a camera implemented by a vehicle in a transportation network, wherein the remote system is external from the vehicle;

receiving, at the remote system, an input indicating at least one of a steering angle or a distance to travel;

determining, at the remote system, a projected path of the vehicle in the transportation network based on the input, wherein the projected path is determined using spatial coordinates relative to the camera;

determining, at the remote system from the spatial coordinates and camera parameters associated with the camera, pixel coordinates in the video stream corresponding to the projected path;

determining, at the remote system, a predicted position of the vehicle in the video stream based on a latency associated with the video stream; and

displaying, at the remote system, an overlay in the video stream based on the pixel coordinates, the overlay including a visualization of the projected path starting from the predicted position.

16 . The non-transitory computer readable medium storing instructions of claim 15 , the operations further comprising:

receiving, at the remote system, a second input indicating a selection of the overlay; and

transmitting, by the remote system, information to the vehicle based on the second input, the information configured to cause the vehicle to follow the projected path.

17 . The non-transitory computer readable medium storing instructions of claim 15 , wherein the camera parameters comprise an orientation of the camera and a resolution of the camera.

18 . The non-transitory computer readable medium storing instructions of claim 15 , wherein the overlay further includes an indication of the latency, and wherein the indication is color coded with a first color for lesser latency, a second color for moderate latency, or a third color for greater latency.

19 . The non-transitory computer readable medium storing instructions of claim 15 , wherein the overlay is represented by lines corresponding to a lane of travel.

20 . The non-transitory computer readable medium storing instructions of claim 15 , wherein the input is received through a graphical user interface (GUI) by a detecting a mouse hovering over a portion of a wheel in the GUI to configure the steering angle and by detecting a position of a slider in the GUI to configure the distance, and wherein a selection of the overlay is received by detecting clicking of a button of at least one of the mouse or a keyboard.