IP Library › Granted Patent US 12,345,542
Granted Patent B2
US 12,345,542 · App. 16/949,932 · Granted Jul 1, 2025

Context sensitive user interface for enhanced vehicle operation

Inventors: Pawel Pietryka (Los Angeles, CA); Jean Helfenstein (Marina Del Rey, CA)
Assignee: Tesla, Inc.
G01C21/367B60K35/00G01C21/3664G06V20/593G06V40/103B60K35/10B60K35/28B60K35/29B60K2360/111B60K2360/115B60K2360/119B60K2360/165B60K2360/166B60K2360/179B60K2360/1868
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,345,542
App. No.
16/949,932
Granted
Jul 1, 2025
Kind
B2
Abstract

Systems and methods for context sensitive user interfaces for enhanced vehicle operation. An example method is method implemented by a system of one or more processors, the system being included in a vehicle. The method includes causing presentation, via a display of the vehicle, of a unified user interface comprising a combined view which aggregates an autonomous visualization and map information, the autonomous visualization and map information being related to a zoom level, with the unified user interface being generated based on first contextual information indicative of operation of the vehicle. Second contextual information indicative of subsequent operation of the vehicle is accessed. The unified user interface is updated based on the second contextual information, with the combined view being adjusted, and with adjusting the combined view including adjusting a size of the combined view in the unified user interface or adjusting the zoom level.

Claims (48)

1. A method implemented by a system of one or more processors, the system being included in a vehicle, and the method comprising:

causing presentation, via a display of the vehicle, of a unified user interface comprising a combined view which aggregates an autonomous visualization and map information, the autonomous visualization and map information being associated with a zoom level, wherein the unified user interface is generated based on first contextual information indicative of operation of the vehicle;

accessing second contextual information indicative of subsequent operation of the vehicle, wherein the first contextual information indicates that the vehicle is navigating along a route, wherein the combined view further aggregates navigation information, and wherein the second contextual information indicates that the vehicle has moved along the route; and

updating the unified user interface based on the second contextual information, wherein the combined view is adjusted, and wherein adjusting the combined view comprises the zoom level,

wherein the route is associated with a plurality of driving events comprising upcoming turns, off-ramps, and on-ramps indicated in the navigation information, and wherein the zoom level is adjusted according to the upcoming driving events,

wherein the autonomous visualization includes a back portion of the vehicle, and wherein the system maintains the autonomous visualization of the back portion of the vehicle while there are less than a threshold number of upcoming driving events, and

wherein based on the route indicating at least the threshold number of upcoming driving events, the combined view is updated to zoom-out,

wherein the combined view includes:

a graphical representation of one or more lanes of a road on which the vehicle is located,

a graphical depiction of the vehicle positioned in a particular lane of the one or more lanes,

and wherein based on a particular upcoming driving event of the threshold number of upcoming driving events indicating a lane change from the particular lane to an adjacent lane, the unified user interface is updated to:

identify the adjacent lane and present an animation depicting the vehicle moving to the adjacent lane, and

identify a different vehicle in the adjacent lane and include information indicating the vehicle is to wait until safe to change to the adjacent lane.

2. The method of claim 1 , wherein the autonomous visualization includes a graphical representation of a real-world environment in which the vehicle is located.

3. The method of claim 2 , wherein the graphical representation of the real-world environment comprises graphical depictions of other vehicles and graphical representations of one or more objects which are proximate to the vehicle.

4. The method of claim 1 , wherein the combined view includes:

map information comprising a graphical representation of a map associated with the road, wherein the graphical depiction of the vehicle is depicted as being on the map.

5. The method of claim 4 , wherein the combined view represents a driving view, and wherein the graphical depiction of the vehicle is animated as driving in the particular lane.

6. The method of claim 1 , wherein the method further comprises:

receiving user input associated with updating the zoom level associated with the combined view;

identifying a portion of the map information based on the updated zoom level; and

updating the combined view, wherein the combined view includes a graphical representation of the portion of the map information.

7. The method of claim 1 , wherein the autonomous visualization comprises a graphical depiction of the vehicle, and wherein the graphical depiction is reduced in size based on the adjusted zoom level.

8. The method of claim 1 , wherein the unified user interface enables control of vehicle functionality, and wherein control of vehicle functionality comprises control of a heating, ventilation, and air conditioning system, or control of a music application, or control of a navigation user interface.

9. The method of claim 8 , wherein the unified user interface includes a plurality of icons associated with respective vehicle functionality, wherein updating the unified user interface comprises:

updating the unified user interface to include a menu associated with a selected icon, wherein the menu is included in a first portion of the unified user interface; and

re-sizing the combined view, wherein the combined view is included in a second portion which is not occluded by the menu.

10. The method of claim 1 , wherein a size of the autonomous visualization is adjusted based on the adjusted zoom level and wherein an area associated with the map information is adjusted based on the adjusted zoom level.

11. The method of claim 1 , wherein the combined view illustrates a first driving event of the plurality of driving events, and wherein based on the combined view being zoomed out, the combined view illustrates a plurality of second driving events of the plurality of driving events.

12. The method of claim 1 , wherein the unified user interface is responsive to user input causing adjustment of the zoom level or translation of the map information along a particular direction.

13. Non-transitory computer storage media storing instructions for execution by a system of one or more processors, the system being included in a vehicle, and the instructions causing the one or more processors to perform operations comprising:

causing presentation, via a display of the vehicle, of a unified user interface comprising a combined view which aggregates an autonomous visualization, map information, and navigation information, wherein the combined view is associated with a zoom level indicating an area of a real-world environment reflected in the combined view, and wherein the vehicle is in a navigation mode associated with a destination;

accessing sensor information from a plurality of sensors, wherein the sensor information comprises a location of the vehicle and images from image sensors positioned about the vehicle; and

determining that the zoom level is to be adjusted based on the sensor information, wherein the combined view is updated to be associated with the adjusted zoom level,

wherein the autonomous visualization includes a back portion of the vehicle, and wherein the system maintains the autonomous visualization of a back portion of the vehicle while there are less than a threshold number of upcoming driving events,

and wherein based on the route indicating at least the threshold number of upcoming driving events the combined view is configured to zoom-out,

wherein the combined view includes:

a graphical representation of one or more lanes of a road on which the vehicle is located,

a graphical depiction of the vehicle positioned in a particular lane of the one or more lanes,

and wherein based on a particular upcoming driving event of the threshold number of upcoming driving events indicating a lane change from the particular lane to an adjacent lane, the unified user interface is updated to:

identify the adjacent lane and present an animation depicting the vehicle moving to the adjacent lane, and

identify a different vehicle in the adjacent lane and include information indicating the vehicle is to wait until safe to change to the adjacent lane.

14. The computer storage media of claim 13 , wherein for the combined view,

the autonomous visualization comprises:

a graphical representation of a real-world environment proximate to the vehicle, the graphical representation being generated based on the images from the image sensors,

the map information comprises:

a graphical representation of a portion of a map associated with the area of the real-world environment, and the navigation information comprises:

one or more graphical representations of the driving events which are associated with the area of the real-world environment.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 14, 2021
From: PIETRYKA, PAWEL; HELFENSTEIN, JEAN
To: TESLA, INC.
Reel/Frame 054926/0431 →
Continuity (3)
Provisional Application 62938842 · Nov 21, 2019
Provisional Application 62938769 · Nov 21, 2019
Related Publication 20210156705A1 · May 27, 2021
References Cited (25)
US 6529822B1 · Millington · 2003 [cited by examiner]
US 9109921B1 · Furio · 2015 [cited by examiner]
US 10464397B2 · Castiglioni et al. · 2019 [cited by applicant]
US 20020126876A1 · Paul et al. · 2002 [cited by applicant]
US 20040243307A1 · Geelen · 2004 [cited by applicant]
US 20130103313A1 · Moore · 2013 [cited by examiner]
US 20130229522A1 · Schofield · 2013 [cited by applicant]
US 20150241240A1 · Yamada · 2015 [cited by examiner]
US 20150369623A1 · Blumenberg · 2015 [cited by examiner]
US 20160178383A1 · Mays · 2016 [cited by examiner]
US 20170111587A1 · Herbst · 2017 [cited by examiner]
US 20170217368A1 · Lewis et al. · 2017 [cited by applicant]
US 20170356755A1 · Strawn · 2017 [cited by examiner]
US 20190094040A1 · Lewis · 2019 [cited by examiner]
US 20190212162A1 · Sata · 2019 [cited by examiner]
US 20190351918A1 · Maeng · 2019 [cited by examiner]
US 20200217687A1 · Camhi · 2020 [cited by examiner]
CN 102742265 · 2012 [cited by applicant]
CN 102999255 · 2013 [cited by applicant]
CN 108349423 · 2018 [cited by applicant]
DE 102010062464 · 2012 [cited by applicant]
DE 102018203462 · 2019 [cited by applicant]
EP 1548404 · 2005 [cited by applicant]
JP 2014178942 · 2014 [cited by applicant]
International Search Report and Written Opinion dated May 4, 2021 in patent application No. PCT/US2020/061543. [cited by applicant]
Cited By (6)
US 1,101,785 US 1,117,294 US 1,118,649 US 1,119,943 US 1,121,641 US 1,121,662