Enhanced object detection for autonomous vehicles based on field view
Systems and methods for enhanced object detection for autonomous vehicles based on field of view. An example method includes obtaining an image from an image sensor of one or more image sensors positioned about a vehicle. A field of view for the image is determined, with the field of view being associated with a vanishing line. A crop portion corresponding to the field of view is generated from the image, with a remaining portion of the image being downsampled. Information associated with detected objects depicted in the image is outputted based on a convolutional neural network, with detecting objects being based on performing a forward pass through the convolutional neural network of the crop portion and the remaining portion.
1. A method implemented by a system of one or more processors, the method comprising:
obtaining an image from an image sensor of one or more image sensors positioned about a vehicle;
determining a field of view for the image, the field of view being associated with a vanishing line;
generating, from the image, a crop portion corresponding to the field of view, and a remaining portion, wherein the remaining portion of the image is downsampled; and
outputting, via a convolutional neural network, information associated with detected objects depicted in the image, wherein detecting objects comprises performing a forward pass through the convolutional neural network of the crop portion and the remaining portion.
2. The method of claim 1 , wherein the output information comprises one or more objects classified in the image and location information associated with the objects, and wherein the output information is configured for use in autonomous driving of the vehicle.
3. The method of claim 1 , wherein a first forward pass through the convolutional network is performed for the crop portion, and wherein a second forward pass through the convolutional network is performed for the remaining portion, and wherein output information is determined based on the first forward pass and the second forward pass.
4. The method of claim 3 , wherein the convolutional neural network detects a first object depicted in the crop portion and a second object depicted in the remaining portion corresponding to a same real-world object, and wherein the method further comprises:
removing, from the output information, one of the first object or the second object.
5. The method of claim 4 , wherein the second object is removed, the second object being associated with less accurate location information than the first object.
6. The method of claim 1 , wherein the vanishing line is determined based on location information of a road depicted in the image.
7. The method of claim 1 , wherein the vanishing line is determined based on map information and inertial measurement unit (IMU) information, and wherein the method comprises:
identifying, based on the IMU, a horizon line; and
determining, based on map information, a vehicle heading, wherein the crop portion represents corresponds with an expected location of the vehicle in a threshold period of time or a threshold distance.
8. A system comprising one or more processors and non-transitory computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
obtaining an image from an image sensor of one or more image sensors positioned about a vehicle;
determining a field of view for the image, the field of view being associated with a vanishing line;
generating, from the image, a crop portion corresponding to the field of view and a remaining portion wherein the remaining portion of the image is downsampled; and
outputting, via a machine learning model, information associated with detected objects depicted in the image.
9. The system of claim 8 , wherein the output information comprises one or more objects classified in the image and location information associated with the objects, and wherein the output information is configured for use in autonomous driving of the vehicle.
10. The system of claim 9 , wherein a first forward pass through the convolutional network is performed for the crop portion, and wherein a second forward pass through the convolutional network is performed for the remaining portion, and wherein output information is determined based on the first forward pass and the second forward pass.
11. The system of claim 10 , wherein the convolutional neural network detects a first object depicted in the crop portion and a second object depicted in the remaining portion corresponding to a same real-world object, and wherein the operations further comprise:
removing, from the output information, one of the first object or the second object.
12. The system of claim 11 , wherein the second object is removed, the second object being associated with less accurate location information than the first object.
13. The system of claim 9 , wherein the vanishing line is determined based on location information of a road depicted in the image.
14. The system of claim 9 , wherein the vanishing line is determined based on map information and inertial measurement unit (IMU) information, and wherein the operations further comprise:
identifying, based on the IMU, a horizon line; and
determining, based on map information, a vehicle heading, wherein the crop portion represents corresponds with an expected location of the vehicle in a threshold period of time or a threshold distance.
15. Non-transitory computer storage media storing instructions that when executed by a system of one or more processors, cause the one or more processors to perform operations comprising:
obtaining an image from an image sensor of one or more image sensors positioned about a vehicle;
determining a field of view for the image, the field of view being associated with a vanishing line;
generating, from the image, a crop portion corresponding to the field of view and a remaining portion, wherein the remaining portion of the image is downsampled; and
outputting, via a machine learning model, information associated with detected objects depicted in the image.
16. The computer storage media of claim 15 , wherein the output information comprises one or more objects classified in the image and location information associated with the objects, and wherein the output information is configured for use in autonomous driving of the vehicle.
17. The computer storage media of claim 15 , wherein a first forward pass through the convolutional network is performed for the crop portion, and wherein a second forward pass through the convolutional network is performed for the remaining portion, and wherein output information is determined based on the first forward pass and the second forward pass.
18. The computer storage media of claim 17 , wherein the convolutional neural network detects a first object depicted in the crop portion and a second object depicted in the remaining portion corresponding to a same real-world object, and wherein the operations further comprise:
removing, from the output information, one of the first object or the second object.
19. The computer storage media of claim 18 , wherein the second object is removed, the second object being associated with less accurate location information than the first object.
20. The system of claim 9 , wherein the vanishing line is determined based on map information and inertial measurement unit (IMU) information, and wherein the operations further comprise:
identifying, based on the IMU, a horizon line; and
determining, based on map information, a vehicle heading, wherein the crop portion represents corresponds with an expected location of the vehicle in a threshold period of time or a threshold distance.