Automatic graphical content recognition for vehicle applications
In various examples, a gaze direction of a user's eyes may be tracked and synced with perception data of the vehicle to determine POIs that the user is interested in. In some examples, POIs may be stored as waypoints in a waypoint catalog or store and included as part of a map. As a user is driving in a vehicle down a roadway, a system onboard the vehicle may access the map to determine locations of the vehicle, and may reference the waypoint catalog to determine the POIs that the vehicle passes. Using an advertiser name, contact information, an advertisement image, advertiser website information, links to additional content, etc. relating to each waypoint, a log of the passed POIs may be stored for access by the user.
1 . A method comprising:
computing, using one or more sensors of a vehicle, a location of the vehicle that corresponds to a position within a map of an environment, the map indicating a plurality of features at feature locations in the map;
determining, based at least on the position within the map, a virtual projection of one or more portions of a field of view associated with a user;
determining that the virtual projection at least partially intersects, in a coordinate space of the map, at least one physical feature external to the vehicle;
based at least on the determination that the virtual projection at least partially intersects the at least one physical feature and the at least one physical feature being mapped to one or more of the feature locations for at least one feature of the plurality of features, selecting the at least one feature from the plurality of features;
based at least on the selecting of the at least one feature, determining one or more content items associated with the at least one feature; and
causing data associated with the one or more content items to be stored in association with an application.
2 . The method of claim 1 , wherein an orientation of the virtual projection is based at least on one or more of:
one or more gaze directions of the user;
one or more gestures of the user;
one or more head movements of the user;
one or more voice inputs from the user; or
one or more eye movements of the user.
3 . The method of claim 1 , wherein the determining that the virtual projection at least partially intersects the at least one physical feature includes, based at least on mapping the virtual projection to the coordinate space of the map, comparing the virtual projection to the one or more of the feature locations in the map.
4 . The method of claim 1 , wherein the determining the virtual projection at least partially intersects the at least one physical feature includes:
determining a bounding shape location corresponding to the at least one physical feature within the coordinate space;
determining an intersection between at least a portion of a virtual volume corresponding to the virtual projection and the bounding shape location within the coordinate space; and
based at least on the determining of the intersection, determining that an amount of overlap between the virtual volume and a bounding shape corresponding to the bounding shape location is greater than a threshold value, wherein the selecting of the at least one feature is based at least on the amount of overlap being greater than the threshold value.
5 . The method of claim 1 , wherein the causing the data to be stored in association with the application is based at least on determining one or more of:
an amount of overlap between the virtual projection and the at least one physical feature;
an amount of time the virtual projection overlaps with the at least one physical feature; or
an actuation, by the user, of a trigger actuator.
6 . The method of claim 1 , wherein the determining the one or more content items includes:
capturing sensor data using an external sensor of the vehicle, the external sensor including a sensor field of view that includes a point of interest (POI) corresponding to the at least one feature;
analyzing one or more portions of the sensor data that corresponds to the POI; and
based at least on the analyzing, determining the one or more content items.
7 . The method of claim 1 , wherein the selecting is further based at least on receiving user input indicating a selection of the at least one feature while the virtual projection at least partially intersects the at least one physical feature.
8 . A system comprising:
one or more processing units to execute operations including:
determining, using one or more sensors of a vehicle, a location of the vehicle that corresponds to a position within a map of an environment, the map indicating a plurality of features at feature locations in the map;
identifying, based at least on the position within the map, a virtual projection of one or more portions of a field of view associated with a user;
determining that the virtual projection at least partially overlaps, in a coordinate space of the map, at least one physical feature external to the vehicle;
based at least on the identifying and the at least one physical feature being mapped to one or more of the feature locations for at least one feature of the plurality of features, selecting the at least one feature from the plurality of features;
based at least on the selecting of the at least one feature, determining one or more content items associated with the at least one feature; and
causing data associated with the one or more content items to be stored in association with an application.
9 . The system of claim 8 , wherein an orientation of the virtual projection is based at least on one or more of:
one or more gaze directions of the user;
one or more gestures of the user;
one or more head movements of the user;
one or more voice inputs from the user; or
one or more eye movements of the user.
10 . The system of claim 8 , wherein the one or more of the feature locations correspond to at least a portion of a virtual indicator of a point of interest (PoI) and the one or more content items are associated with the PoI.
11 . The system of claim 8 , wherein the determining that the virtual projection at least partially overlaps includes:
determining a bounding shape location corresponding to the at least one physical feature within the coordinate space; and
determining an overlap between the virtual projection and the bounding shape location within the coordinate space.
12 . The system of claim 8 , wherein the causing the data to be stored in association with the application is based at least on determining one or more of:
an amount of overlap between the virtual projection and the at least one physical feature;
an amount of time the virtual projection overlaps with the at least one physical feature; or
an actuation, by the user, of a trigger actuator.
13 . The system of claim 8 , wherein the determining the one or more content items includes:
capturing sensor data using an external sensor of the vehicle, the external sensor including a sensor field of view that includes a point of interest (POI) corresponding to the at least one feature;
analyzing one or more portions of the sensor data that corresponds to the POI; and
based at least on the analyzing, determining the one or more content items.
14 . The system of claim 8 , wherein an overlap between the virtual projection and the at least one physical feature is identified in the coordinate space of the map using a mapping of the virtual projection to the coordinate space.
15 . At least one processor comprising:
one or more circuits to cause data associated with one or more content items corresponding to at least one feature of a plurality of features indicated at feature locations in a map of an environment to be stored in association with an application based at least on:
a location of a vehicle that corresponds to a position within the map being computed using one or more sensors of a vehicle,
based at least on the position within the map, a first determination of a virtual projection of one or more portions of a field of view associated with a user,
a second determination that the virtual projection at least partially intersects, in a coordinate space of the map, at least one physical feature external to the vehicle, and
a selection of the at least one feature from the plurality of features based at least on the second determination and the at least one physical feature being mapped to one or more of the feature locations for the at least one feature.
16 . The processor of claim 15 , wherein an orientation of the virtual projection is based at least on one or more of:
one or more gaze directions of the user;
one or more gestures of the user;
one or more head movements of the user;
one or more voice inputs from the user; or
one or more eye movements of the user.
17 . The processor of claim 15 , wherein the at least one feature corresponds to at least a portion of a virtual indicator of a point of interest (PoI) and the one or more content items are associated with the PoI.
18 . The processor of claim 15 , wherein the second determination is based at least on:
determining a bounding shape location corresponding to the at least one physical feature within the coordinate space; and
determining an overlap between the virtual projection and the bounding shape location within the coordinate space.
19 . The processor of claim 15 , wherein the causing the data to be stored in association with the application is based at least on one or more of:
an amount of overlap between the virtual projection and the at least one physical feature;
an amount of time the virtual projection overlaps with the at least one physical feature; or
an actuation, by the user, of a trigger actuator.
20 . The processor of claim 15 , wherein the virtual projection is identified in the coordinate space of the map using a mapping of the virtual projection to the coordinate space.