Enhanced three-dimensional training data generation
Systems, methods, and non-transitory computer readable media configured to generate enhanced three-dimensional information. Three-dimensional information of a scene may be obtained. The three-dimensional information may define a three-dimensional point cloud model of the scene. The three-dimensional information may be determined based on distances of the scene from a location. Image information may be obtained. The image information may define one or more images of an object. The object may be identified based on the image information. A three-dimensional point cloud model of the object may be obtained. Enhanced three-dimensional information of the scene may be generated by inserting the three-dimensional point cloud model of the object into the three-dimensional point cloud model of the scene.
1. A system comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to perform:
obtaining three-dimensional information of a scene, the three-dimensional information defining a three-dimensional point cloud model of the scene, the three-dimensional information determined based on distances of the scene from a location;
obtaining image information, the image information defining one or more images of an object;
identifying the object based on the image information;
obtaining a three-dimensional point cloud model of the object; and
generating enhanced three-dimensional information of the scene by inserting the three-dimensional point cloud model of the object into the three-dimensional point cloud model of the scene, wherein the inserting includes:
identifying a portion of the three-dimensional point cloud model of the scene corresponding to the object; and
replacing the portion of the three-dimensional point cloud model of the scene corresponding to the object with the three-dimensional point cloud model of the object.
2. The system of claim 1 , wherein the enhanced three-dimensional information of the scene is used to train a machine learning model.
3. The system of claim 2 , wherein the machine learning model is used to control motions of a vehicle.
4. The system of claim 1 , wherein the distances of the scene from the location are measured using LIDAR.
5. The system of claim 4 , wherein the image information is captured by one or more image capture devices concurrently with the measurements of the distances of the scene using LIDAR.
6. The system of claim 1 , wherein the object includes a person, an animal, a vehicle, or a structure.
7. The system of claim 1 , wherein a point density of the three-dimensional point cloud model of the object inserted into the three-dimensional point cloud model of the scene is determined based on a given distance of the object from the location.
8. The system of claim 1 , wherein the three-dimensional point cloud model of the object is inserted into the three-dimensional point cloud model of the scene based on an orientation of the object within the scene.
9. A method implemented by a computing system including one or more processors and storage media storing machine-readable instructions, wherein the method is performed using the one or more processors, the method comprising:
obtaining three-dimensional information of a scene, the three-dimensional information defining a three-dimensional point cloud model of the scene, the three-dimensional information determined based on distances of the scene from a location;
obtaining image information, the image information defining one or more images of an object;
identifying the object based on the image information;
obtaining a three-dimensional point cloud model of the object; and
generating enhanced three-dimensional information of the scene by inserting the three-dimensional point cloud model of the object into the three-dimensional point cloud model of the scene, wherein the inserting includes:
identifying a portion of the three-dimensional point cloud model of the scene corresponding to the object; and
replacing the portion of the three-dimensional point cloud model of the scene corresponding to the object with the three-dimensional point cloud model of the object.
10. The method of claim 9 , wherein the enhanced three-dimensional information of the scene is used to train a machine learning model.
11. The method of claim 10 , wherein the machine learning model is used to control motions of a vehicle.
12. The method of claim 9 , wherein the distances of the scene from the location are measured using LIDAR.
13. The method of claim 12 , wherein the image information is captured by one or more image capture devices concurrently with the measurements of the distances of the scene using LIDAR.
14. The method of claim 9 , wherein the object includes a person, an animal, a vehicle, or a structure.
15. The method of claim 9 , wherein a point density of the three-dimensional point cloud model of the object inserted into the three-dimensional point cloud model of the scene is determine based on a given distance of the object from the location.
16. The method of claim 9 , wherein the three-dimensional point cloud model of the object is inserted into the three-dimensional point cloud model of the scene based on an orientation of the object within the scene.
17. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:
obtaining three-dimensional information of a scene, the three-dimensional information defining a three-dimensional point cloud model of the scene, the three-dimensional information determined based on distances of the scene from a location;
obtaining image information, the image information defining one or more images of an object;
identifying the object based on the image information;
obtaining a three-dimensional point cloud model of the object; and
generating enhanced three-dimensional information of the scene by inserting the three-dimensional point cloud model of the object into the three-dimensional point cloud model of the scene, wherein the inserting includes:
identifying a portion of the three-dimensional point cloud model of the scene corresponding to the object; and
replacing the portion of the three-dimensional point cloud model of the scene corresponding to the object with the three-dimensional point cloud model of the object.
18. The non-transitory computer readable medium of claim 17 , wherein a point density of the three-dimensional point cloud model of the object inserted into the three-dimensional point cloud model of the scene is determined based on a given distance of the object from the location.
19. The system of claim 1 , wherein a point density of the three-dimensional point cloud model of the object is uniform.
20. The system of claim 1 , wherein a point density of the three-dimensional point cloud model of the object is non-uniform with a first one or more portions of the object being represented with higher resolution data points in the three-dimensional point cloud model of the object and a second one or more portions of the object being represented with lower resolution data points in the three-dimensional point cloud model of the object.