Low- and high-fidelity classifiers applied to road-scene images
Disclosures herein teach applying a set of sections spanning a down-sampled version of an image of a road-scene to a low-fidelity classifier to determine a set of candidate sections for depicting one or more objects in a set of classes. The set of candidate sections of the down-sampled version may be mapped to a set of potential sectors in a high-fidelity version of the image. A high-fidelity classifier may be used to vet the set of potential sectors, determining the presence of one or more objects from the set of classes. The low-fidelity classifier may include a first Convolution Neural Network (CNN) trained on a first training set of down-sampled versions of cropped images of objects in the set of classes. Similarly, the high-fidelity classifier may include a second CNN trained on a second training set of high-fidelity versions of cropped images of objects in the set of classes.
1. A method comprising:
calculating a maximum factor by which an image can be down-sampled while maintaining a threshold ratio of entropy in the down-sampled version of the image;
down-sampling the image by the maximum factor to generate the down-sampled version of the image;
processing the down-sampled image with a low-fidelity classifier, wherein the low-fidelity classifier is trained to calculate a probability that a zone of the down-sampled version of the image comprises an object;
identifying a probable zone having a high probability for comprising the object;
forward feeding the probable zone to a high-fidelity classifier, wherein the high-fidelity classifier is trained to confirm a presence of the object in the probable zone.
2. The method of claim 1 , wherein down-sampling the image to generate the down-sampled version of the image comprises down-sampling a plurality of zones of the image to generate a plurality of down-sampled zones of the image.
3. The method of claim 1 , wherein the probable zone is identified based on the down-sampled version of the image and wherein the method further comprises mapping the probable zone to a higher-resolution version of the image.
4. The method of claim 3 , wherein forward feeding the probable zone to the high-fidelity classifier comprises forward feeding the probable zone in the higher-resolution version of the image.
5. The method of claim 1 , wherein calculating the maximum factor by which the image can be down-sampled to generate the down-sampled version of the image comprises determining the threshold ratio of entropy, wherein the threshold ratio of entropy is determined based one or more of:
maintaining an absolute value of entropy in the image;
maintaining a predetermine percent of entropy in the image; or
maintaining at least eighty percent of entropy in the image.
6. The method of claim 1 , further comprising extracting the plurality of zones from the image by sliding a window over the image pixel-by-pixel such that each zone of the plurality of zones is the size of the window and overlaps surrounding zones.
7. The method of claim 6 , further comprising:
abstracting a set of scaled zones from the down-sampled version of the image, wherein scaled zones in the set of scaled zones have different dimensions than the sliding window;
selecting a scaled zone from the set of scaled zones for which the low-fidelity classifier indicates a probability of an existence of a scaled object; and
mapping the scaled zone to a scaled sector of the high-resolution version.
8. The method of claim 1 , wherein the low-fidelity classifier and the high-fidelity classifier are each portions of a same convolutional neural network.
9. The method of claim 1 , wherein the image is of a scene surrounding a vehicle and wherein the method further comprises:
determining whether the object exists in the scene surrounding the vehicle based on whether the presence of the object is confirmed by the high-fidelity classifier;
determining a driving maneuver for the vehicle based on whether the object exists in the scene surrounding the vehicle; and
providing the driving maneuver to an automated driving system of the vehicle to be executed by the automated driving system.
10. The method of claim 1 , further comprising:
cropping each of a plurality of images that are labeled as comprising the object;
down-sampling each of the plurality of images to generate a down-sampled set of images;
training the low-fidelity classifier with the down-sampled set of images; and
training the high-fidelity classifier with at least one of the plurality of images at full resolution.
11. A system comprising:
a vehicle;
a camera affixed to the vehicle configured to capture an image of a scene surrounding the vehicle; and
a processor in communication with the camera and programmable to execute instructions stored in non-transitory computer readable storage media, the instructions comprising:
calculating a maximum factory by which the image can be down-sampled while maintaining a threshold ratio of entropy in the down-sampled version of the image;
down-sampling the image by the maximum factor to generate the down-sampled version of the image;
processing the image with a low-fidelity classifier, wherein the low-fidelity classifier is trained to calculate a probability, for each of a plurality of zones of the image, that a zone of the down-sampled versions of the image comprises an object;
identifying a probable zone having a high probability for comprising the object;
forward feeding the probable zone to a high-fidelity classifier, wherein the high-fidelity classifier is trained to confirm a presence of the object in the probable zone.
12. The system of claim 11 , wherein the instructions are such that
down-sampling the image to generate the down-sampled version of the image
comprises down-sampling a plurality of zones of the image to generate a plurality of down-sampled zones of the image.
13. The system of claim 11 , wherein the probable zone is identified based on the down-sampled version of the image and wherein the method further comprises mapping the probable zone to a higher-resolution version of the image.
14. The system of claim 13 , wherein forward feeding the probable zone to the high-fidelity classifier comprises forward feeding the probable zone in the higher-resolution version of the image.
15. The system of claim 11 , wherein the instructions further comprise calculating the maximum factor by which the image can be down-sampled to generate the down-sampled version of the image comprises determining the threshold ratio of entropy, wherein the threshold ratio of entropy is determined based one or more of:
maintaining an absolute value of entropy in the image;
maintaining a predetermine percent of entropy in the image; or
maintaining at least eighty percent of entropy in the image.
16. A processor that is programmable to execute instructions stored in non-transitory computer readable storage media, the instructions comprising:
calculating a maximum factor by which an image can be down-sampled while maintaining a threshold ratio of entropy in the down-sampled version of the image;
down-sampling the image by the maximum factor to generate the down-sampled version of the image;
processing the down-sampled image with a low-fidelity classifier, wherein the low-fidelity classifier is trained to calculate a probability that a zone of the down-sampled version of the image comprises an object;
identifying a probable zone having a high probability for comprising the object;
forward feeding the probable zone to a high-fidelity classifier, wherein the high-fidelity classifier is trained to confirm a presence of the object in the probable zone.
17. The processor of claim 16 , wherein the instructions are such that
down-sampling the image to generate the down-sampled version of the image
comprises down-sampling a plurality of zones of the image to generate a plurality of down-sampled zones of the image.
18. The processor of claim 16 , wherein the probable zone is identified based on the down-sampled version of the image and wherein the method further comprises mapping the probable zone to a higher-resolution version of the image.
19. The processor of claim 17 , further comprising
extracting the plurality of zones from the image by sliding a window over the image pixel-by-pixel such that each zone of the plurality of zones is the size of the window and overlaps surrounding zones.
20. The processor of claim 16 , wherein the low-fidelity classifier and the high-fidelity classifier are each portions of a same convolutional neural network.