Contextual augmentation of map information using overlays
Systems, methods, and non-transitory computer readable media are provided for displaying and annotating map-based geolocation data at an augmented reality (AR) headset. Users with access to the map-based geolocation data can create or confirm annotations for geospatial data that may be sent to the server computer and transmitted back to the headset of the user as well as different AR headsets associated with other users.
1 . A computer-implemented method, comprising:
providing, on a first augmented reality (AR) device, a first user interface for a first user using the first AR device to generate first feedback of an image of a location at a first time observed through the first AR device;
receiving the first feedback from the first AR device, the first feedback identifying an entity;
in response to receiving the first feedback, translating the first feedback into a first annotation;
transmitting, from the first AR device, an indication of the first annotation to a second AR device;
providing, on the second AR device, a second user interface for a second user using the second AR device to generate second feedback of the image of the location at a second time, the second feedback identifying an attribute of the entity that was absent from the first feedback;
in response to receiving the second feedback from the second AR device, translating the second feedback into a second annotation;
correlating the first annotation corresponding to the first AR device and the second annotation corresponding to the second AR device as annotations of the location, wherein the first feedback or the second feedback relates to a shape of a path at the location, and the first annotation or the second annotation indicates any traversable or non-traversable portion of a previously unlabeled path based on potential sizes, types, or compositions of an entity attempting to traverse the any traversable or non-traversable portion; and
outputting, on the first AR device, the correlated annotations of the location.
2 . The computer-implemented method of claim 1 , further comprising:
collecting the image of the location and the correlated annotations as training data for training a classifier machine learning model, wherein the correlated annotations are used labels for the location.
3 . The computer-implemented method of claim 2 , further comprising:
obtaining time information associated with the image of the location and the correlated annotations; and
including the time information in the training data such that the classifier machine learning model is able to compensate for different image characteristics of the location that arise due to timing differences.
4 . The computer-implemented method of claim 1 , wherein the correlating of the first annotation from the first AR device and the second annotation from the second AR device as annotations of the location comprises:
merging a first mark from the first AR device and a second mark from the second AR device as the annotations of the location.
5 . The computer-implemented method of claim 1 , wherein the first AR device and the second AR device face the location from different perspectives, and the second AR device observes objects at the location that are invisible by the first AR device.
6 . The computer-implemented method of claim 5 , wherein the outputting, on the first AR device, the correlated annotations of the location as an overlay with respect to the location comprises:
displaying the objects observed by the second AR device on the first AR device.
7 . The computer-implemented method of claim 1 , wherein the first feedback indicates a degree of traversability of a route corresponding to unmarked terrain.
8 . The computer-implemented method of claim 1 , wherein the first AR device and the second AR device have different scores based on reliabilities of the first AR device and the second AR device; and the first annotation and the second annotation have different weights corresponding to the different scores.
9 . The computer-implemented method of claim 1 , wherein the second feedback was obtained following an event that transformed the entity; and the computer-implemented method further comprises:
storing the first feedback and the second feedback as time-series data.
10 . The computer-implemented method of claim 9 , further comprising ingesting the time-series data into a machine-learning model to train the machine learning model.
11 . The computer-implemented method of claim 1 , wherein the second feedback is associated with a different environmental resolution compared to the first feedback.
12 . The computer-implemented method of claim 1 , wherein the visually emphasizing comprises visually emphasizing one or more demarcations of the entity.
13 . The computer-implemented method of claim 1 , wherein the second feedback identifies a different entity that was absent from the first feedback and invisible from a perspective of the first AR device; and
the outputting of the correlated annotations comprises outputting the different entity in response to the first AR device facing in a direction towards the different entity but being obstructed from seeing the different entity.
14 . The computer-implemented method of claim 1 , wherein the outputting, on the first AR device, the correlated annotations of the location comprises generating a topological skeleton and outputting the correlated annotations over the topological skeleton.
15 . The computer-implemented method of claim 1 , wherein a non-traversable portion comprises a restricted region and a buffer around the restricted region.
16 . A computing system comprising:
one or more processors;
a memory storing instructions that, when executed by the one or more processors, cause the computing system to perform:
providing, on a first augmented reality (AR) device, a first user interface for a first user using the first AR device to generate first feedback of an image of a location at a first time observed through the first AR device;
receiving the first feedback from the first AR device, the first feedback identifying an entity;
in response to receiving the first feedback, translating the first feedback into a first annotation;
transmitting, from the first AR device, an indication of the first annotation to a second AR device;
providing, on the second AR device, a second user interface for a second user using the second AR device to generate second feedback of the image of the location at a second time, the second feedback identifying an attribute of the entity that was absent from the first feedback;
in response to receiving the second feedback from the second AR device, translating the second feedback into a second annotation;
correlating the first annotation corresponding to the first AR device and the second annotation corresponding to the second AR device as annotations of the location, wherein the first feedback or the second feedback relates to a shape of a path at the location, and the first annotation or the second annotation indicates any traversable or non-traversable portion of a previously unlabeled path based on potential sizes, types, or compositions of an entity attempting to traverse the any traversable or non-traversable portion; and
outputting, on the first AR device, the correlated annotations of the location.
17 . The computing system of claim 16 , wherein the second AR device is at a remote location from the location observed through the first AR device.
18 . The computing system of claim 16 , wherein the first AR device is an AR headset, and the second AR device is an unmanned aerial vehicle (UAV).
19 . The computing system of claim 16 , wherein the instructions further cause the computing system to perform:
collecting the image of the location and the correlated annotations as training data for training a classifier machine learning model, wherein the correlated annotations are used labels for the location.
20 . The computing system of claim 19 , wherein the instructions further cause the computing system to perform:
obtaining time information associated with the image of the location and the correlated annotations; and
including the time information in the training data such that the classifier machine learning model is able to compensate for different image characteristics of the location that arise due to timing differences.