Augmenting real sensor recordings with simulated sensor data
Original sensor data is received from one or more sensors of a vehicle. Free space around the vehicle is identified according to the sensor data, such as by identifying regions where data points have a height below a threshold. A location for an object model is selected from the free space. A plane is fitted to sensor data around the location and the object model is oriented according to an orientation of the plane. Sensing of the object model by a sensor of the vehicle is simulated to obtain simulated data, which is then added to the original sensor data. Sensor data corresponding to objects that would have been obscured by the object model is removed from the original sensor data. Augmented sensor data may be used to validate a control algorithm or train a machine learning model.
1. A method comprising, by a computing device:
receiving sensor data from a controller of a vehicle;
identifying available space around the vehicle in the sensor data;
placing an object model at a relative location to the vehicle;
simulating sensing of the object model to obtain simulated sensor data; and
adding the simulated sensor data to the sensor data to obtain augmented sensor data.
2. The method of claim 1 , comprising:
identifying, by the computing device, a portion of the sensor data that would not be detectable due to placement of the object model at the relative location; and
removing, by the computing device, the portion of the sensor data from the augmented sensor data.
3. The method of claim 2 , further comprising:
defining, by the computing device, a shadow cone of the object model with respect to a sensor location on the vehicle; and
searching, by the computing device, for the portion of the sensor data that would not be detectable within the shadow cone.
4. The method of claim 1 , further comprising:
identifying, by the computing device, an area of the available space around the relative location;
fitting, by the computing device, a plane to the area; and
transforming, by the computing device, the object model according to an orientation of the plane.
5. The method of claim 4 , further comprising:
identifying, by the computing device, the area as being covered by the object model;
determining, by the computing device, (a) that the area covers insufficient data points of the sensor data to define the plane; and
in response to determining (a), expanding, by the computing device, the area until the area covers sufficient data points to fit the plane.
6. The method of claim 1 , wherein identifying the available space around the vehicle in the sensor data comprises:
evaluating, by the computing device, a height associated data points in the sensor data; and
identifying, by the computing device, as the available space, regions in which the height associated with data points in the regions are less than a threshold height.
7. The method of claim 6 , wherein evaluating the height associated with data points in the sensor data comprises defining, by the computing device, the height with respect to a horizontal plane of the vehicle.
8. The method of claim 1 , wherein the sensor data includes at least one of LIDAR (light detection and ranging), RADAR (radio detection and ranging), and image data.
9. The method of claim 1 , further comprising testing, by the computing device, one of a detection algorithm and a control algorithm according to the augmented sensor data.
10. The method of claim 1 , further comprising training, by the computing device, a machine learning model to at least one of detect obstacles and remove objects according to the augmented sensor data.
11. A system comprising one or more processing devices and one or more memory devices operably coupled to the one or more processing devices, the one or more memory devices storing executable code effective to cause the one or more processing devices to:
receive sensor data from a controller of a vehicle;
identify available space around the vehicle in the sensor data;
place an object model at a relative location to the vehicle;
simulate sensing of the object model to obtain simulated sensor data; and
add the simulated sensor data to the sensor data to obtain augmented sensor data.
12. The system of claim 11 , wherein the executable code is further effective to cause the one or more processing devices to:
identify a portion of the sensor data that would not be detectable due to placement of the object model at the relative location; and
remove the portion of the sensor data from the augmented sensor data.
13. The system of claim 12 , wherein the executable code is further effective to cause the one or more processing devices to:
define a shadow cone of the object model with respect to a sensor location on the vehicle; and
search for the portion of the sensor data that would not be detectable within the shadow cone.
14. The system of claim 11 , wherein the executable code is further effective to cause the one or more processing devices to:
identify an area of the available space around the relative location;
fit a plane to the area; and
transform the object model according to an orientation of the plane.
15. The system of claim 14 , wherein the executable code is further effective to cause the one or more processing devices to:
identify the area as being covered by the object model;
evaluate whether (a) the area covers insufficient data points of the sensor data to define the plane; and
if (a), expand the area until the area covers sufficient data points to fit the plane.
16. The system of claim 11 , wherein the executable code is further effective to cause the one or more processing devices to:
evaluate a height associated data points in the sensor data; and
identify, as the available space, regions in which the height associated with data points in the regions are less than a threshold height.
17. The system of claim 16 , wherein the executable code is further effective to cause the one or more processing devices to evaluate the height associated with data points in the sensor data by defining the height with respect to a horizontal plane of the vehicle.
18. The system of claim 11 , wherein the sensor data includes at least one of LIDAR (light detection and ranging), RADAR (radio detection and ranging), and image data.
19. The system of claim 11 , wherein the executable code is further effective to cause the one or more processing devices to test at least one of a detection algorithm and a control algorithm according to the augmented sensor data.
20. The system of claim 11 , wherein the executable code is further effective to cause the one or more processing devices to train a machine learning model to at least one of detect obstacles and remove objects according to the augmented sensor data.