Automatic graphical content recognition for vehicle applications

1. A method comprising:

determining a location of a vehicle within an environment using map data;

determining, using one or more internal sensors of the vehicle, a gaze direction of a user, the gaze direction extending to one or more points in the environment external to the vehicle;

based at least on the location of the vehicle, mapping the gaze direction of the user to the map data, the map data including a plurality of waypoints at waypoint locations;

based at least on the mapping, determining the gaze direction is fixated on a point of interest (PoI) corresponding to the one or more points in the environment based at least on a waypoint from the plurality of waypoints corresponding to the PoI, the waypoint estimated to be within a field of view of the user;

determining one or more content items associated with the waypoint; and

causing data associated with the one or more content items to be stored in association with an application.

2. The method of claim 1 , further comprising:

projecting a virtual representation of the gaze direction to the one or more points in the environment external to the vehicle; and

based at least on the projecting, determining the PoI at least partially overlaps with the one or more points in the environment,

wherein the determining the gaze direction is fixated on the PoI is based at least on the determining the PoI at least partially overlaps with the one or more points in the environment.

3. The method of claim 2 , further comprising:

determining an actuation, by the user, of a trigger actuator, wherein the causing data associated with the one or more content items to be stored in association with the application is based at least on the actuation occurring while the PoI is at least partially overlapping the one or more points in the environment.

4. The method of claim 1 , wherein the map data comprises a high definition (HD) map, and the mapping the gaze direction to the HD map includes comparing a virtual representation of the field of view of the user with the HD map.

5. The method of claim 1 , wherein the determining the one or more content items includes:

capturing sensor data using an external sensor of the vehicle, the external sensor including a sensor field of view that includes a point of interest (POI) corresponding to the waypoint;

analyzing the POI; and

based at least on the analyzing, determining the one or more content items.

6. The method of claim 1 , wherein the determining the one or more content items includes:

accessing a catalog of waypoints; and

determining, from the catalog, the one or more content items associated with the waypoint.

7. The method of claim 1 , further comprising:

receiving a request to view the data associated with the one or more content items; and

causing display of the data using the application executed by at least one of the vehicle or a client device separate from the vehicle.

8. A method comprising:

identifying, based at least on first sensor data generated using at least one external sensor of a vehicle, one or more content items associated with an identified point of interest (POI) in a field of view of at least one external sensor;

generating, based at least on second sensor data generated using at least one internal sensor of a vehicle, a virtual representation of a gaze direction of an occupant of the vehicle, the gaze direction extending to one or more points in an environment external to the vehicle;

determining, using the virtual representation of the gaze direction, that the one or more points in the environment at least partially overlap with the identified POI for a threshold period of time; and

based at least on the determining, causing data associated with the one or more content items to be stored in association with an application.

9. The method of claim 8 , further comprising:

determining that a trigger action is executed by the occupant,

wherein the causing the data to be stored is based at least on the trigger action.

10. The method of claim 8 , wherein the at least one internal sensor or the at least one external sensor include at least one of a camera, a LIDAR sensor, a RADAR sensor, a microphone, a GNSS sensor, an IMU sensor, or an ultrasonic sensor.

11. The method of claim 8 , wherein the identifying the one or more content items includes:

localizing the vehicle to a high definition (HD) map based at least on the first sensor data;

determining the identified POI based at least on the localizing; and

determining the one or more content items associated with a waypoint corresponding to the POI from a catalog of content items.

12. The method of claim 8 , wherein the threshold period of time includes at least one of a sequential period of time or a cumulative period of time over a time window.

13. The method of claim 8 , wherein the determining that the one or more points in the environment at least partially overlap with the identified POI for the threshold period of time includes:

determining a bounding shape location corresponding to the identified POI within a coordinate space;

projecting the virtual representation of the gaze direction into the coordinate space; and

determining an overlap amount between the projected virtual representation of the gaze direction and the bounding shape location within the coordinate space.

14. A system comprising:

one or more processing units to execute operations including:

determining a location of a vehicle within a map;

determining, using one or more internal sensors of the vehicle, a gaze direction of a user, the gaze direction extending to one or more points in an environment external to the vehicle;

based at least on the location of the vehicle, mapping the gaze direction of the user to the map, the map including a plurality of waypoints at waypoint locations;

based at least on the mapping, determining the gaze direction is fixated on a point of interest (PoI) corresponding to the one or more points in the environment based at least on a waypoint from the plurality of waypoints corresponding to the PoI, the waypoint estimated to be within a field of view of the user;

determining one or more content items associated with the waypoint; and

causing data associated with the one or more content items to be stored in association with an application.

15. The system of claim 14 , wherein the operations further comprise:

determining, based at least on the one or more internal sensors, that a trigger action is executed by the user,

wherein the causing the data to be stored is based at least on the trigger action.

16. The system of claim 14 , wherein the one or more internal sensors of the vehicle includes a camera.

17. The system of claim 14 , wherein the map is a high definition (HD) map, and the mapping the gaze direction to the HD map includes comparing a virtual representation of the field of view of the user with the HD map.

18. The system of claim 14 , wherein the determining the one or more content items includes:

capturing sensor data using an external sensor of the vehicle, the external sensor including a sensor field of view that includes a point of interest (POI) corresponding to the waypoint;

analyzing the POI; and

based at least on the analyzing, determining the one or more content items.

19. The system of claim 14 , wherein the determining the one or more content items includes:

accessing a catalog of waypoints; and

determining, from the catalog, the one or more content items associated with the waypoint.

20. The system of claim 14 , wherein the operations further comprise:

receiving a request to view the data associated with the one or more content items; and

causing display of the data using the application executed by at least one of the vehicle or a client device separate from the vehicle.