Encoding LiDAR scanned data for generating high definition maps for autonomous vehicles
Embodiments relate to methods for efficiently encoding sensor data captured by an autonomous vehicle and building a high definition map using the encoded sensor data. The sensor data can be LiDAR data which is expressed as multiple image representations. Image representations that include important LiDAR data undergo a lossless compression while image representations that include LiDAR data that is more error-tolerant undergo a lossy compression. Therefore, the compressed sensor data can be transmitted to an online system for building a high definition map. When building a high definition map, entities, such as road signs and road lines, are constructed such that when encoded and compressed, the high definition map consumes less storage space. The positions of entities are expressed in relation to a reference centerline in the high definition map. Therefore, each position of an entity can be expressed in fewer numerical digits in comparison to conventional methods.
1. A method comprising:
receiving, from a sensor mounted on a vehicle, sensor data describing locations within an environment around the vehicle captured by the sensor;
determining based on the received sensor data a plurality of sensor orientation datasets, each sensor orientation dataset comprising a pitch value, a yaw value, a range value, and an intensity value;
constructing one or more image representations from the plurality of sensor orientation datasets, comprising, for each image representation, repeating for a plurality of pixels:
identifying, in the image representation, a pixel location corresponding to a combination of a pitch value and a yaw value of a sensor orientation dataset, and
determining a value of the identified pixel from one of the range value or the intensity value of the sensor orientation dataset, and
setting the determined value for the pixel at the identified pixel location;
compressing each of the one or more image representations to generate compressed codes representing the plurality of sensor orientation datasets; and
transmitting the compressed codes for generation of a high definition map of the environment around the vehicle, the high definition map for use in driving by one or more autonomous vehicles.
2. The method of claim 1 , wherein a number of rows of each of the one or more image representations is dependent on a number of lasers of the vehicle sensor.
3. The method of claim 1 , wherein a number of columns of each of the one or more image representations is dependent on a minimum angular resolution of the vehicle sensor.
4. The method of claim 1 , wherein constructing one or more image representations comprises:
generating a first image representation, wherein a value of a pixel of the first image representation is a most significant byte of the range value of the dataset;
generating a second image representation, wherein a value of a pixel of the second image representation is a least significant byte of the range value of the dataset; and
generating a third image representation, wherein a value of a pixel of the third image representation is a byte of the intensity value of the dataset.
5. The method of claim 4 , wherein a step value of the most significant byte and least significant byte of the range value is dependent on a maximum range detectable by the vehicle sensor.
6. The method of claim 4 , wherein constructing one or more image representations of the three dimensional data further comprises:
generating a fourth image representation, wherein a value of a pixel of the fourth image representation is a color value.
7. The method of claim 4 , wherein encoding each of the one or more image representations comprises encoding the first image representation using a lossless encoding process.
8. The method of claim 4 , wherein encoding each of the one or more image representations comprises encoding the second image representation and the third image representation using a lossy encoding process.
9. The method of claim 1 , wherein constructing one or more image representations of the three dimensional data further comprises:
for each of the one or more image representations, identifying one or more pixels of the image representation with a value indicating missing data; and
for each of the one or more pixels with a non-existent value, modifying the pixel with the value indicating missing data to store a value based on other pixels of the image representation.
10. The method of claim 9 , further comprising:
generating a bitmap indicating a presence of one or more pixels in the one or more image representations that has the value indicating missing data;
encoding the generated bitmap; and
transmitting the encoded bitmap, the encoded bitmap further used for generation of the high definition map of the environment around the vehicle.
11. A non-transitory computer readable storage medium comprising computer instructions that, when executed by a processor, cause the processor to:
receive, from a sensor mounted on a vehicle, sensor data describing locations within an environment around the vehicle captured by the sensor;
determining based on the received sensor data, a plurality of sensor orientation datasets, each sensor orientation dataset comprising a pitch value, a yaw value, a range value, and an intensity value;
construct one or more image representations from the plurality of sensor orientation datasets, comprising, for each image representation, repeating for a plurality of pixels:
identify, in the image representation, a pixel location corresponding to a combination of a pitch value and a yaw value of a sensor orientation dataset, and
determine a value of the identified pixel from one of the range value or the intensity value of the sensor orientation dataset, and
set the determined value for the pixel at the identified pixel location;
compress each of the one or more image representations to generate compressed codes representing the plurality of sensor orientation datasets; and
transmit the compressed codes for generation of a high definition map of the environment around the vehicle, the high definition map for use in driving by one or more autonomous vehicles.
12. The non-transitory computer storage of claim 11 , wherein a number of rows of each of the one or more image representations is dependent on a number of lasers of the vehicle sensor.
13. The non-transitory computer storage of claim 11 , wherein a number of columns of each of the one or more image representations is dependent on a minimum angular resolution of the vehicle sensor.
14. The non-transitory computer storage of claim 11 , wherein the computer instructions that cause the processor to construct one or more image representations further comprise instructions that, when executed by the processor, cause the processor to:
generate a first image representation, wherein a value of a pixel of the first image representation is a most significant byte of the range value of the dataset;
generate a second image representation, wherein a value of a pixel of the second image representation is a least significant byte of the range value of the dataset; and
generate a third image representation, wherein a value of a pixel of the third image representation is a byte of the intensity value of the dataset.
15. The non-transitory computer storage of claim 14 , wherein a step value of the most significant byte and least significant byte of the range value is dependent on a maximum range detectable by the vehicle sensor.
16. The non-transitory computer storage of claim 14 , wherein the computer instructions that cause the processor to encode each of the one or more image representations further comprise computer instructions that, when executed by the processor, cause the processor to encode the first image representation using a lossless encoding process.
17. The non-transitory computer storage of claim 14 , wherein the computer instructions that cause the processor to encode each of the one or more image representations further comprise computer instructions that, when executed by the processor, cause the processor to encode the second image representation and the third image representation using a lossy encoding process.
18. The non-transitory computer storage of claim 11 , wherein the computer instructions that cause the processor to construct one or more image representations of the three dimensional data further comprises instructions that, when executed by the processor, cause the processor to:
for each of the one or more image representations, identify one or more pixels of the image representation with a value indicating missing data; and
for each of the one or more pixels with a non-existent value, modify the pixel with the value indicating missing data to store a value based on other pixels of the image representation.
19. The non-transitory computer storage of claim 18 , further comprising computer instructions that, when executed by the processor, cause the processor to:
generate a bitmap indicating a presence of one or more pixels in the one or more image representations that has the value indicating missing data;
encode the generated bitmap; and
transmit the encoded bitmap, the encoded bitmap further used for generation of the high definition map of the environment around the vehicle.
20. A computer-system comprising:
a processor; and
a non-transitory computer readable storage medium comprising computer instructions that, when executed by a processor, cause the processor to:
receive, from a sensor mounted on a vehicle, sensor data describing locations within an environment around the vehicle captured by the sensor;
determine based on the received sensor data, a plurality of sensor orientation datasets, each sensor orientation dataset comprising a pitch value, a yaw value, a range value, and an intensity value;
construct one or more image representations from the plurality of sensor orientation datasets, comprising, for each image representation, repeating for a plurality of pixels:
identify, in the image representation, a pixel location corresponding to a combination of a pitch value and a yaw value of a sensor orientation dataset, and
determine a value of the identified pixel from one of the range value or the intensity value of the sensor orientation dataset, and
set the determined value for the pixel at the identified pixel location;
compress each of the one or more image representations to generate compressed codes representing the plurality of sensor orientation datasets; and
transmit the compressed codes for generation of a high definition map of the environment around the vehicle, the high definition map for use in driving by one or more autonomous vehicles.
21. The computer-system of claim 20 , wherein a number of rows of each of the one or more image representations is dependent on a number of lasers of the vehicle sensor.
22. The computer-system of claim 20 , wherein a number of columns of each of the one or more image representations is dependent on a minimum angular resolution of the vehicle sensor.
23. The computer-system of claim 20 , wherein the computer instructions that cause the processor to construct one or more image representations further comprise instructions that, when executed by the processor, cause the processor to:
generate a first image representation, wherein a value of a pixel of the first image representation is a most significant byte of the range value of the dataset;
generate a second image representation, wherein a value of a pixel of the second image representation is a least significant byte of the range value of the dataset; and
generate a third image representation, wherein a value of a pixel of the third image representation is a byte of the intensity value of the dataset.
24. The computer-system of claim 23 , wherein a step value of the most significant byte and least significant byte of the range value is dependent on a maximum range detectable by the vehicle sensor.
25. The computer-system of claim 23 , wherein the computer instructions that cause the processor to encode each of the one or more image representations further comprise computer instructions that, when executed by the processor, cause the processor to encode the first image representation using a lossless encoding process.
26. The computer-system of claim 23 , wherein the computer instructions that cause the processor to encode each of the one or more image representations further comprise computer instructions that, when executed by the processor, cause the processor to encode the second image representation and the third image representation using a lossy encoding process.
27. The computer-system of claim 20 , wherein the computer instructions that cause the processor to construct one or more image representations of the three dimensional data further comprise instructions that, when executed by the processor, cause the processor to:
for each of the one or more image representations, identify one or more pixels of the image representation with a value indicating missing data; and
for each of the one or more pixels with a non-existent value, modify the pixel with the value indicating missing data to store a value based on other pixels of the image representation.
28. The computer-system of claim 27 , further comprising computer instructions that, when executed by the processor, cause the processor to:
generate a bitmap indicating a presence of one or more pixels in the one or more image representations that has the value indicating missing data;
encode the generated bitmap; and
transmit the encoded bitmap, the encoded bitmap further used for generation of the high definition map of the environment around the vehicle.