System and method for vehicle position and velocity estimation based on camera and lidar data
A vehicle position and velocity estimation based on camera and LIDAR data are disclosed. A particular embodiment includes: receiving input object data from a subsystem of an autonomous vehicle, the input object data including image data from an image generating device and distance data from a distance measuring device; determining a two-dimensional (2D) position of a proximate object near the autonomous vehicle using the image data received from the image generating device; tracking a three-dimensional (3D) position of the proximate object using the distance data received from the distance measuring device over a plurality of cycles and generating tracking data; determining a 3D position of the proximate object using the 2D position, the distance data received from the distance measuring device, and the tracking data; determining a velocity of the proximate object using the 3D position and the tracking data; and outputting the 3D position and velocity of the proximate object relative to the autonomous vehicle.
1. A system comprising:
a data processor; and
a vehicle position and velocity estimation module, executable by the data processor, the vehicle position and velocity estimation module being configured to perform a proximate object position and velocity estimation operation for an autonomous vehicle, the proximate object position and velocity estimation operation being configured to:
receive input object data from a subsystem of the autonomous vehicle, the input object data including image data from an image generating device and distance data from a distance measuring device, the distance measuring device being one or more light imaging, detection, and ranging (LIDAR) sensors;
determine a two-dimensional (2D) position of a proximate object near the autonomous vehicle using the image data received from the image generating device and semantic segmentation processing of the image data;
track a three-dimensional (3D) position of the proximate object using the distance data received from the distance measuring device over a plurality of cycles and generate tracking data;
correlate the proximate object identified from the image data with the proximate object identified and tracked from the distance data, the correlation being configured to match the 2D position of the proximate object detected in the image data with the 3D position of the same proximate object detected in the distance data;
determine a 3D position of the proximate object using the 2D position, the distance data received from the distance measuring device, and the tracking data;
determine a velocity of the proximate object using the 3D position and the tracking data; and
output the 3D position and velocity of the proximate object relative to the autonomous vehicle.
2. The system of claim 1 wherein the image generating device is one or more cameras.
3. The system of claim 1 wherein the proximate object position and velocity estimation operation being further configured to retain position tracking data from a previous cycle.
4. The system of claim 1 wherein the image data corresponds to at least one frame from a video stream generated by one or more cameras.
5. The system of claim 1 wherein the distance data corresponds to a point cloud generated by the one or more LIDAR sensors.
6. The system of claim 1 wherein the 3D position and velocity of the proximate object is output to a vehicle control subsystem causing the autonomous vehicle to avoid the proximate object.
7. The system of claim 1 wherein the proximate object position and velocity estimation operation being configured to generate the 3D position and velocity of the proximate object even when intermittent, errant, or unstable image data or distance data is received.
8. A method comprising:
receiving input object data from a subsystem of an autonomous vehicle, the input object data including image data from an image generating device and distance data from a distance measuring device, the distance measuring device being one or more light imaging, detection, and ranging (LIDAR) sensors;
determining a two-dimensional (2D) position of a proximate object near the autonomous vehicle using the image data received from the image generating device and semantic segmentation processing of the image data;
tracking a three-dimensional (3D) position of the proximate object using the distance data received from the distance measuring device over a plurality of cycles and generating tracking data;
correlating the proximate object identified from the image data with the proximate object identified and tracked from the distance data, the correlating including matching the 2D position of the proximate object detected in the image data with the 3D position of the same proximate object detected in the distance data;
determining a 3D position of the proximate object using the 2D position, the distance data received from the distance measuring device, and the tracking data;
determining a velocity of the proximate object using the 3D position and the tracking data; and
outputting the 3D position and velocity of the proximate object relative to the autonomous vehicle.
9. The method of claim 8 wherein the image generating device is one or more cameras.
10. The method of claim 8 including retaining position tracking data from a previous cycle.
11. The method of claim 8 wherein the image data corresponds to at least one frame from a video stream generated by one or more cameras.
12. The method of claim 8 wherein the distance data corresponds to a point cloud generated by the one or more LIDAR sensors.
13. The method of claim 8 wherein the 3D position and velocity of the proximate object is output to a vehicle control subsystem causing the autonomous vehicle to avoid the proximate object.
14. The method of claim 8 including generating the 3D position and velocity of the proximate object even when intermittent, errant, or unstable image data or distance data is received.
15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive input object data from a subsystem of an autonomous vehicle, the input object data including image data from an image generating device and distance data from a distance measuring device, the distance measuring device being one or more light imaging, detection, and ranging (LIDAR) sensors;
determine a two-dimensional (2D) position of a proximate object near the autonomous vehicle using the image data received from the image generating device and semantic segmentation processing of the image data;
track a three-dimensional (3D) position of the proximate object using the distance data received from the distance measuring device over a plurality of cycles and generate tracking data;
correlate the proximate object identified from the image data with the proximate object identified and tracked from the distance data, the correlation being configured to match the 2D position of the proximate object detected in the image data with the 3D position of the same proximate object detected in the distance data;
determine a 3D position of the proximate object using the 2D position, the distance data received from the distance measuring device, and the tracking data;
determine a velocity of the proximate object using the 3D position and the tracking data; and
output the 3D position and velocity of the proximate object relative to the autonomous vehicle.
16. The non-transitory machine-useable storage medium of claim 15 wherein the proximate object position and velocity estimation operation being configured to generate the 3D position and velocity of the proximate object even when intermittent, errant, or unstable image data or distance data is received.
17. The non-transitory machine-useable storage medium of claim 15 wherein the 3D position and velocity of the proximate object is output to a vehicle control subsystem causing the autonomous vehicle to avoid the proximate object.