Systems and methods for object detection at various ranges using multiple range imagery
Systems and methods are directed to object detection at various ranges for autonomous vehicles. In one example, a system includes a camera providing a first field of view; a machine-learned model that has been trained to generate object detection range estimates based at least in part on labeled training data representing image data having a second field of view different from the first field of view; and a computing system including one or more processors; and memory including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations include obtaining image data from the camera. inputting the image data from the camera to the machine-learned model; obtaining a first range estimate as an output of the machine-learned model, wherein the first range estimate represents estimates for the second field of view; generating transformed range estimate by applying a range estimate transform to the first range estimate output from the machine-learned model; and providing the transformed range estimate for use in controlling operation of an autonomous vehicle.
1. A system comprising:
a camera providing a first field of view;
a machine-learned model that has been trained to generate object detection range estimates based at least in part on training data representing image data having a second field of view different from the first field of view; and
a computing system comprising:
one or more processors; and
memory including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
obtaining image data from the camera;
inputting the image data from the camera to the machine-learned model;
obtaining a first range estimate as an output of the machine-learned model, wherein the first range estimate represents estimates for the second field of view;
generating a transformed range estimate by applying a range estimate transform to the first range estimate output from the machine-learned model, wherein the range estimate transform transforms the first range estimate into the transformed range estimate based at least in part on a ratio of focal lengths between the second field of view associated with the training data and the first field of view associated with the camera; and
providing the transformed range estimate for use in controlling operation of an autonomous vehicle.
2. The system of claim 1 wherein the range estimate transform provides a transformation from a second field of view range to a first field of view range.
3. The system of claim 1 wherein the image data from the camera comprises image data that depicts objects in front of the autonomous vehicle.
4. The system of claim 1 wherein the transformed range estimate provides range estimates for detected objects at ranges greater than 100 meters.
5. The system of claim 1 wherein the image data from the camera providing the first field of view comprises an enlarged view of an area within the second field of view.
6. The system of claim 1 wherein the training data comprises objects in the image data having the second field of view labeled with ground-truth range data generated by a LIDAR system.
7. The system of claim 1 wherein the camera comprises a narrow field of view camera, the first field of view comprises a narrow field of view, the training data comprises image data that has a wide view of view, and the training data comprises image data collected by a wide field of view camera.
8. A computer-implemented method for object detection comprising:
obtaining, by a computing system comprising one or more computing devices, image data from a camera providing a first field of view;
inputting, by the computing system, the image data from the camera to a machine-learned model that has been trained to generate object detection range estimates based at least in part on training data representing image data having a second field of view different from the first field of view:
obtaining, by the computing system, a first range estimate as an output of the machine-learned model, wherein the first range estimate represents an estimate for the second field of view;
generating, by the computing system, a transformed range estimate by applying a range estimate transform to the first range estimate output from the machine-learned model, wherein the range estimate transform transforms the first range estimate into the transformed range estimate based at least in part on a ratio of focal lengths between the second field of view associated with the training data and the first field of view associated with the camera; and
providing, by the computing system, the transformed range estimate to a motion planning system for use in determining operation of an autonomous vehicle.
9. The computer-implemented method of claim 8 wherein the range estimate transform provides a transformation from a second field of view range to a first field of view range.
10. The computer-implemented method of claim 8 wherein the image data from the camera comprises image data that depicts objects in front of the autonomous vehicle.
11. The computer-implemented method of claim 8 wherein the transformed range estimate provides a range estimate for a detected objects at a range greater than 100 meters.
12. The computer-implemented method of claim 8 wherein the image data from the camera providing the first field of view comprises an enlarged view of an area within the second field of view.
13. The computer-implemented method of claim 8 wherein the training data comprises objects in image data having the second field of view labeled with ground-truth range data generated by a LIDAR system.
14. An autonomous vehicle comprising:
a camera providing a first field of view;
a machine-learned model that has been trained to generate object detection range estimates based at least in part on training data representing image data having a second field of view different from the first field of view; and
a vehicle computing system comprising:
one or more processors; and
memory including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
obtaining image data from the camera;
inputting the image data to the machine-learned model;
obtaining a first range estimate as an output of the machine-learned model, wherein the first range estimate represents an estimate for the second field of view;
generating a transformed range estimate by applying a range estimate transform to the first range estimate output from the machine-learned model, wherein the range estimate transform transforms the first range estimate into the transformed range estimate based at least in part on a ratio of focal lengths between the second field of view associated with the training data and the first field of view associated with the camera; and
providing the transformed range estimate to a motion planning system for use in determining operation of the autonomous vehicle.
15. The autonomous vehicle of claim 14 wherein the range estimate transform provides a transformation from a second field of view range to a first field of view range.
16. The autonomous vehicle of claim 14 wherein the image data from the camera providing the first field of view comprises an enlarged view of an area within the second field of view.
17. The autonomous vehicle of claim 14 wherein the training data comprises objects in image data having the second field of view labeled with ground-truth range data generated by a LIDAR system.