IP Library › Granted Patent US 12709298
Granted Patent B2
US 12709298 · App. 18/821,367 · Granted Aug 18, 2026

System to communicate vehicle intent using projection

Inventors: Erik Lee St. Gray (Tacoma, WA); Marcell Jose Vazquez-Chanlatte (Palo Alto, CA); Mark Bailey (Seattle, WA)
Assignee: Nissan North America, Inc.
B60W60/0015B60Q1/507B60W30/0956G06F40/56G06V20/56G06V40/10G06V40/28B60Q2400/50B60Q2800/10B60W2420/403
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12709298
App. No.
18/821,367
Granted
Aug 18, 2026
Kind
B2
Abstract

Communicating vehicle intent using projection includes receiving an exception scene generated using scene data captured by one or more systems of a vehicle. The exception scene is indicated by a detection of at least one potential hazard to a safe operation of the vehicle, and the vehicle includes a projector. A hazard type associated with the potential hazard(s) and one or more gestures are determined using the exception scene. A drive plan for the vehicle based on the hazard type and the one or more gestures is generated. A representation of the drive plan in an area external of the vehicle is displayed, using the projector of the vehicle, in a visually perceptible manner. The drive plan is executed by a control system of the vehicle.

Claims (38)

1 . A method, comprising:

receiving an exception scene generated using scene data captured by one or more systems of a vehicle, wherein the exception scene is indicated by a detection of at least one potential hazard to a safe operation of the vehicle and the vehicle comprises a projector;

determining a hazard type associated with the at least one potential hazard and one or more gestures using the exception scene, wherein the hazard type is at least one of an electrical hazard, a fire hazard, a water hazard, a fallen object hazard, a crash hazard, or an unknown hazard;

generating a drive plan for the vehicle based on the hazard type and the one or more gestures;

displaying, using the projector, a representation of the drive plan in an area external to the vehicle in a visually perceptible manner, wherein the representation of the drive plan includes a hazard symbol that visually represents the hazard type and a path symbol that visually represents a motion to be taken by the vehicle to safely navigate the exception scene; and

executing, by a control system of the vehicle, the drive plan.

2 . The method of claim 1 , wherein the one or more gestures are hand and body movements detected within the exception scene and the one or more gestures provide a direction for navigating the exception scene.

3 . The method of claim 1 , wherein the hazard type is an unknown hazard and responsive to the hazard type being an unknown hazard a symbol representing confusion is associated with the hazard type.

4 . The method of claim 1 , wherein the drive plan includes one or more actions, wherein the one or more actions allow the vehicle to safely navigate the exception scene.

5 . The method of claim 4 , wherein generating the drive plan comprises:

associating at least one of the one or more actions with a symbol that visually represents a motion to be taken by the vehicle to safely navigate the exception scene.

6 . The method of claim 4 , wherein the drive plan includes an interactive question.

7 . The method of claim 6 , wherein the interactive question is generated using a large language model (LLM).

8 . The method of claim 1 , wherein an outer boundary of the area exterior to the vehicle is between 1 and 15 meters from the vehicle.

9 . An apparatus, comprising:

a memory subsystem; and

one or more processors configured to execute instructions stored in the memory subsystem to:

receive an exception scene generated using scene data captured by one or more systems of a vehicle, wherein the exception scene is indicated by a detection of at least one potential hazard to a safe operation of the vehicle and the vehicle comprises a projector;

determine a hazard type associated with the at least one potential hazard and one or more gestures using the exception scene, wherein the hazard type is at least one of an electrical hazard, a fire hazard, a water hazard, a fallen object hazard, a crash hazard, or an unknown hazard;

generate a drive plan for the vehicle based on the hazard type and the one or more gestures;

display, using the projector, a representation of the drive plan in an area external to the vehicle in a visually perceptible manner, wherein the representation of the drive plan includes a hazard symbol that visually represents the hazard type and a path symbol that visually represents a motion to be taken by the vehicle to safely navigate the exception scene; and

execute, by a control system of the vehicle, the drive plan.

10 . The apparatus of claim 9 , wherein the one or more gestures are hand and body movements detected within the exception scene and the one or more gestures provide a direction for navigating the exception scene.

11 . The apparatus of claim 9 , wherein the drive plan includes one or more actions, wherein the one or more actions allow the vehicle to safely navigate the exception scene.

12 . The apparatus of claim 11 , wherein generating the drive plan comprises:

associating at least one of the one or more actions with a symbol that visually represents a motion to be taken by the vehicle to safely navigate the exception scene.

13 . The apparatus of claim 11 , wherein the drive plan includes an interactive question generated using a large language model (LLM).

14 . A non-transitory computer-readable medium storing instructions operable to cause one or more processors to perform operations comprising:

receiving an exception scene generated using scene data captured by one or more systems of a vehicle, wherein the exception scene is indicated by a detection of at least one potential hazard to a safe operation of the vehicle and the vehicle comprises a projector;

determining a hazard type associated with the at least one potential hazard and one or more gestures using the exception scene, wherein the hazard type is at least one of an electrical hazard, a fire hazard, a water hazard, a fallen object hazard, a crash hazard, or an unknown hazard;

generating a drive plan for the vehicle based on the hazard type and the one or more gestures;

displaying, using the projector, a representation of the drive plan in an area external to the vehicle in a visually perceptible manner, wherein the representation of the drive plan includes a hazard symbol that visually represents the hazard type and a path symbol that visually represents a motion to be taken by the vehicle to safely navigate the exception scene; and

executing, by a control system of the vehicle, the drive plan.

15 . The non-transitory computer-readable medium of claim 14 , wherein the drive plan includes one or more actions that allow the vehicle to safely navigate the exception scene and generating the drive plan comprises:

associating at least one of the one or more actions with a symbol that visually represents a motion to be taken by the vehicle to safely navigate the exception scene.

16 . The non-transitory computer-readable medium of claim 14 , wherein the one or more gestures are hand and body movements detected within the exception scene and the one or more gestures provide a direction for navigating the exception scene.

17 . The non-transitory computer-readable medium of claim 14 , wherein the drive plan includes an interactive question.

18 . The non-transitory computer-readable medium of claim 17 , wherein the interactive question is generated using a large language model (LLM).