Storing map data based on metadata
Techniques are discussed for generating, maintaining, and accessing maps to be used for vehicle localization and navigation. A map of a region may include map data representing discrete tiles of the region, and be associated with a map manifest listing pointers to content of the map. In some cases, the map manifest may have a recursive structure which references other map manifests. The map files may be stored on a storage system that avoids duplication and allows efficient determination of changes to content. For example, the stored map files may each be associated with a content-based hash value, which may be incorporated into a filename or URI of the map file. Computing systems on a vehicle may use comparison of the content-based hash values to determine files that have changed and need to be downloaded. The computing systems may also use hash values to determine accuracy of downloaded files.
1 . A system comprising:
one or more processors;
a memory; and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
receiving a map manifest representing a map of a first geographic region comprising geospatial tiles, wherein the map manifest includes a first identifier comprising a first hash value indicative of first content of a first map data file associated with a tile of the geospatial tiles;
accessing, from the memory, a second identifier comprising a second hash value indicative of second content of a second map data file associated with the tile;
determining, based on the first identifier not matching the second identifier, a change in map data associated with the tile;
identifying, based at least in part on a textual representation of the first identifier, a storage location of the first map data file, wherein a filename of the storage location includes the textual representation;
based on determining the change, retrieving, from the storage location, the first content of the first map data file to a third map data file in the memory, the third map data file storing the first content;
associating the tile with the third map data file;
receiving an additional map manifest associated with a second geographic region;
determining, based on the map manifest and the additional map manifest, a third map manifest associated with a third geographic region comprising the first geographic region and the second geographic region,
wherein the third map manifest is determined without accessing the first content of the first map data file or the first content of the third map data file; and
controlling, based at least in part on the third map manifest, an autonomous vehicle to traverse the third geographic region.
2 . The system of claim 1 , wherein retrieving the first map data file further comprises:
determining, based on the first content, a first checksum;
determining, based on a third content of the first map data file as retrieved at a first time, a second checksum;
determining that the first checksum does not match the second checksum; and
based on determining that the first checksum does not match the second checksum, retrieving the first map data file at a second time after the first time.
3 . The system of claim 1 , wherein the tile corresponds to a polygon of a three-dimensional mesh representation of the first geographic region or a cell of a two-dimensional tile representation of the first geographic region.
4 . The system of claim 1 , wherein the map manifest is a first map manifest and the map is a first map comprising first geospatial tiles, the operations further comprising:
accessing, from the memory, a second map manifest representing a second map of a fourth geographic region, the fourth geographic region including one or more tiles of the first geospatial tiles; and
generating, based on the second map manifest and the first map manifest, a fourth map manifest representing a combination of the first geographic region and the fourth geographic region.
5 . The system of claim 1 , wherein the memory is on-board the autonomous vehicle, and controlling the autonomous vehicle further comprises:
performing an action of the autonomous vehicle based at least in part on the first content, wherein the action includes at least one of:
a localization action to determine a position of the autonomous vehicle in the third geographic region, or
a planning action to determine a trajectory of the autonomous vehicle for traversing the third geographic region.
6 . A method comprising:
receiving a first map manifest associated with a geographic region, wherein the first map manifest includes a first identifier comprising a hash value determined by providing, as input to a hash function, first content of a first data file associated with a first portion of the geographic region;
determining, based on comparing the first identifier with a second identifier generated based on second content of a second data file associated with the first portion of the geographic region, a change in map data associated with the first portion of the geographic region;
identifying, based at least in part on a first textual representation of the first identifier, a storage location of the first data file, wherein a second textual representation of a Universal Resource Identifier (URI) of the storage location includes the first textual representation;
based on determining the change, retrieving, from the storage location associated with the first identifier, the first content of the first data file to a third data file, the third data file storing the first content;
associating the first portion of the geographic region with the third data file;
receiving a second map manifest associated with a second portion of the geographic region; and
determining, based on the first map manifest and the second map manifest, and without accessing the first content, a third map manifest associated with a third portion of the geographic region comprising the first portion of the geographic region and the second portion of the geographic region,
wherein a vehicle is controlled in the geographic region based at least in part on the third map manifest.
7 . The method of claim 6 , wherein the first content comprises first map data representing the first portion of the geographic region, the method further comprising:
performing an action of the vehicle based at least in part on the first map data from the third data file, wherein the action includes at least one of a localization action or a planning action,
wherein the vehicle is controlled in accordance with the action.
8 . The method of claim 6 , wherein the map data comprises at least one of: three-dimensional mesh data, two-dimensional tile data, intensity data, road network data, radio-frequency (RF) data, or altitude data.
9 . The method of claim 6 , wherein the first data file contains three-dimensional mesh data, and the first identifier is determined by providing, as input, the three-dimensional mesh data to the hash function, and receiving, as output, the first identifier.
10 . The method of claim 6 , wherein the first data file contains two-dimensional tile data, and the first identifier is determined by providing, as input, the two-dimensional tile data to the hash function.
11 . The method of claim 6 , wherein the first portion of the geographic region comprises first geospatial tiles, and the first data file contains an additional map manifest associated with one or more geospatial tiles of the first geospatial tiles.
12 . The method of claim 6 , further comprising:
determining, by inputting content of the third data file to the hash function, a third identifier;
determining that the third identifier matches the first identifier; and
based on determining that the third identifier matches the first identifier, validating an accuracy of the third data file.
13 . The system of claim 1 , wherein the tile is a first tile, and the map manifest includes a sub-manifest associated with one or more second tiles of the geospatial tiles.
14 . The system of claim 1 , wherein the geospatial tiles are first geospatial tiles, and wherein:
the additional map manifest includes an additional value indicative of additional content associated with second geospatial tiles of the second geographic region, and
the third map manifest is further determined without accessing the additional content.
15 . The system of claim 1 , wherein the first hash value is determined by providing, as input to a hash function, the first map data file, the operations further comprising:
determining, by inputting the third map data file to the hash function, a third hash value;
determining a difference between an additional textual representation of the third hash value and at least a portion of the textual representation of the first identifier;
determining, based on the difference, that the first content was retrieved incorrectly; and
based on determining that the first content was retrieved incorrectly, redownloading the first content to the third map data file.
16 . The method of claim 6 , further comprising:
determining, by inputting the third data file to the hash function, an additional hash value;
determining a match between a third textual representation of the additional hash value and at least a portion of the first textual representation; and
determining, based on the match, that the third data file is verified as storing the first content.
17 . One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:
receiving first map metadata associated with a first geographic region of an environment, the first map metadata including one or more first hash values indicative of content of respective map data files associated with the first geographic region;
determining, based on comparing the one or more first hash values with one or more second hash values in a second map metadata associated with the first geographic region, a change in a content of a map data file associated with the one or more second hash values;
based on determining the change, retrieving the content of the map data file to a local map data file associated with a vehicle,
wherein a first textual representation of a Universal Resource Identifier (URI) of the map data file includes a second textual representation of a hash value indicative of the content of the map data file;
associating the first geographic region and the local map data file;
receiving an additional map metadata associated with a second geographic region of the environment;
determining, based on the first map metadata and the additional map metadata, a third map metadata associated with a third geographic region comprising the first geographic region and the second geographic region,
wherein the third map metadata is determined without accessing the content of the map data file or the local map data file; and
controlling, based at least in part on the third map metadata, the vehicle to traverse the third geographic region.
18 . The one or more non-transitory computer-readable media of claim 17 , wherein the map data file is associated with a first geospatial tile of the first geographic region, the operations further comprising:
determining, a third hash value of the one or more second hash values matching the hash value indicative of the content of the map data file, the third hash value associated with a second geospatial tile of the first geographic region; and
associating the second geospatial tile with the map data file.
19 . The one or more non-transitory computer-readable media of claim 18 , wherein the third hash value is a based on Secure Hash Algorithm (SHA), and the Universal Resource Identifier (URI) of the map data file indicates a storage location of the map data file.
20 . The one or more non-transitory computer-readable media of claim 17 , wherein the respective map data files contain one or more of: geospatial tile data, road network data, intensity data, or altitude data.