Vision-based detection and classification of traffic lights
The present disclosure is directed to an autonomous vehicle having a vehicle control system. The vehicle control system includes an image processing system. The image processing system receives an image that includes a plurality of image portions. The image processing system also calculates a score for each image portion. The score indicates a level of confidence that a given image portion represents an illuminated component of a traffic light. The image processing system further identifies one or more candidate portions from among the plurality of image portions. Additionally, the image processing system determines that a particular candidate portion represents an illuminated component of a traffic light using a classifier. Further, the image processing system provides instructions to control the autonomous vehicle based on the particular candidate portion representing an illuminated component of a traffic light.
1. A method, comprising:
receiving, from an image-capture device coupled to a vehicle, an image of a field of view of the vehicle, wherein the image comprises a plurality of image portions;
calculating, for each given image portion of the plurality of image portions, a respective score based on color values of pixels in a first area within the given image portion and color values of pixels in a second area within the given image portion, wherein the score indicates a level of confidence that the given image portion represents an illuminated component of a traffic light;
identifying, from among the plurality of image portions, a particular image portion having a score that exceeds a threshold score;
determining, using a classifier, that the particular image portion represents a traffic light in a particular state; and
providing an indication of the particular state of the traffic light.
2. The method of claim 1 , wherein the classifier has been trained using training data indicative of a plurality of example images that include traffic lights in different states and a plurality of example images that do not include traffic lights.
3. The method of claim 1 , wherein determining, using the classifier, that the particular image portion represents a traffic light in a particular state comprises:
determining, using the classifier, that the particular image portion represents a traffic light with a red light illuminated.
4. The method of claim 1 , wherein determining, using the classifier, that the particular image portion represents a traffic light in a particular state comprises:
determining, using the classifier, that the particular image portion represents a traffic light with a green light illuminated.
5. The method of claim 1 , wherein determining, using the classifier, that the particular image portion represents a traffic light in a particular state comprises:
determining, using the classifier, a shape and color of an illuminated component of the traffic light.
6. The method of claim 1 , further comprising:
determining, using the classifier, a confidence level that the particular image portion represents a traffic light in the particular state.
7. The method of claim 6 , further comprising:
determining that the confidence level exceeds a threshold level of confidence; and
responsive to the confidence level exceeding the threshold level of confidence, providing the indication of the particular state of the traffic light.
8. An image processing system comprising:
an image-capture device coupled to a vehicle;
at least one processor; and
a memory having stored thereon instructions that, upon execution by the at least one processor, cause the image processing system to perform functions comprising:
receiving, from the image-capture device, an image of a field of view of the vehicle, wherein the image comprises a plurality of image portions;
calculating, for each given image portion of the plurality of image portions, a respective score based on color values of pixels in a first area within the given image portion and color values of pixels in a second area within the given image portion, wherein the score indicates a level of confidence that the given image portion represents an illuminated component of a traffic light;
identifying, from among the plurality of image portions, a particular image portion having a score that exceeds a threshold score;
determining, using a classifier, that the particular image portion represents a traffic light in a particular state; and
providing an indication of the particular state of the traffic light.
9. The image processing system of claim 8 , wherein the classifier has been trained using training data indicative of a plurality of example images that include traffic lights in different states and a plurality of example images that do not include traffic lights.
10. The image processing system of claim 8 , wherein determining, using the classifier, that the particular image portion represents a traffic light in a particular state comprises:
determining, using the classifier, that the particular image portion represents a traffic light with a red light illuminated.
11. The image processing system of claim 8 , wherein determining, using the classifier, that the particular image portion represents a traffic light in a particular state comprises:
determining, using the classifier, that the particular image portion represents a traffic light with a green light illuminated.
12. The image processing system of claim 8 , wherein determining, using the classifier, that the particular image portion represents a traffic light in a particular state comprises:
determining, using the classifier, a shape and color of an illuminated component of the traffic light.
13. The image processing system of claim 8 , wherein the functions further comprise:
determining, using the classifier, a confidence level that the particular image portion represents a traffic light in the particular state.
14. The image processing system of claim 13 , wherein the functions further comprise:
determining that the confidence level exceeds a threshold level of confidence; and
responsive to the confidence level exceeding the threshold level of confidence, providing the indication of the particular state of the traffic light.
15. A non-transitory computer-readable medium having stored thereon instructions that, upon execution by at least one processor of a computing device, cause the computing device to perform functions comprising:
receiving, from an image-capture device coupled to a vehicle, an image of a field of view of the vehicle, wherein the image comprises a plurality of image portions;
calculating, for each given image portion of the plurality of image portions, a respective score based on color values of pixels in a first area within the given image portion and color values of pixels in a second area within the given image portion, wherein the score indicates a level of confidence that the given image portion represents an illuminated component of a traffic light;
identifying, from among the plurality of image portions, a particular image portion having a score that exceeds a threshold score;
determining, using a classifier, that the particular image portion represents a traffic light in a particular state; and
providing an indication of the particular state of the traffic light.
16. The non-transitory computer-readable medium of claim 15 , wherein the classifier has been trained using training data indicative of a plurality of example images that include traffic lights in different states and a plurality of example images that do not include traffic lights.
17. The non-transitory computer-readable medium of claim 15 , wherein determining, using the classifier, that the particular image portion represents a traffic light in a particular state comprises:
determining, using the classifier, that the particular image portion represents a traffic light with a red light illuminated.
18. The non-transitory computer-readable medium of claim 15 , wherein determining, using the classifier, that the particular image portion represents a traffic light in a particular state comprises:
determining, using the classifier, that the particular image portion represents a traffic light with a green light illuminated.
19. The non-transitory computer-readable medium of claim 15 , wherein determining, using the classifier, that the particular image portion represents a traffic light in a particular state comprises:
determining, using the classifier, a shape and color of an illuminated component of the traffic light.
20. The non-transitory computer-readable medium of claim 15 , wherein the functions further comprise:
determining, using the classifier, a confidence level that the particular image portion represents a traffic light in the particular state.
21. The non-transitory computer-readable medium of claim 20 , wherein the functions further comprise:
determining that the confidence level exceeds a threshold level of confidence; and
responsive to the confidence level exceeding the threshold level of confidence, providing the indication of the particular state of the traffic light.