Bulb mask representation for traffic light classification
Systems and methods are provided for representing a traffic signal device. The method includes receiving a digital image of a traffic signal device that includes one or more traffic signal elements, representing the traffic signal device as a raster image, each traffic signal element of the traffic signal device being represented by a mask corresponding to a location of the traffic signal element on the traffic signal device, representing each mask in a channel in the raster image, providing the raster image as an input to a neural network to classify a state for each of the one or more traffic signal elements, and receiving, from the neural network, a classified raster image, in which the classified raster image includes a plurality of masks, each mask representing a state of one of the one or more traffic signal elements.
1. A method for representing a traffic signal device, the method comprising:
by a computer vision system of a vehicle, receiving a digital image of a traffic signal device that includes one or more traffic signal elements; and
by a processor:
representing the traffic signal device as a raster image, wherein:
each traffic signal element of the traffic signal device is represented by a mask corresponding to a location of the traffic signal element on the traffic signal device,
each mask is represented in only one of a plurality of channels in the raster image, and
each channel in the raster image is a color channel that indicates a color, feature or both of the traffic signal element represented in the mask;
providing the raster image as an input to a neural network to classify a state of each of the one or more traffic signal elements; and
receiving, from the neural network, a classified raster image, in which the classified raster image includes a plurality of masks, wherein each mask represents the state of one of the one or more traffic signal elements.
2. The method of claim 1 , wherein representing the traffic signal device as a raster image comprises using map data to directly generate the raster image.
3. The method of claim 1 , wherein representing the traffic signal device as a raster image comprises:
generating a digital image from map data; and
converting the digital image to a raster image.
4. The method of claim 1 , wherein the plurality of color channels in the raster image correspond to a plurality of different colors.
5. The method of claim 1 , wherein each traffic signal element of the traffic signal device corresponds to a designated light fixture configured to transmit traffic instructions to one or more drivers.
6. The method of claim 1 , wherein the color, feature, or both of each of the one or more traffic signal elements represented in the color channels comprise one or more of the following:
a green light;
a yellow light;
a red light;
a circular light;
a left arrow light;
a right arrow light;
a forward arrow light;
a light having an arrow in any direction;
a flashing green light;
a flashing yellow light;
a flashing red light;
a U-turn light;
a bicycle light; or
an X-light.
7. The method of claim 1 , wherein each mask corresponds to a discrete region of pixels in the raster image.
8. The method of claim 7 , wherein each discrete region of pixels is rectangular in shape.
9. The method of claim 1 , further comprising:
identifying a face of the traffic signal device, wherein each traffic signal element is located within the face of the traffic signal device.
10. The method of claim 1 , further comprising:
generating a confidence value that a traffic signal element of a traffic signal device correlates to the state; and
if the confidence value is greater than a threshold value, determining that the traffic signal element is in the state.
11. The method of claim 1 , further comprising using a position and shape of each mask in the raster image to identify an angle of the traffic signal device.
12. A system for representing a traffic signal device, the system comprising:
a computer vision system configured to receive a digital image of a traffic signal device that includes one or more traffic signal elements;
a transceiver configured to send and receive digital information; and
a processor and program instructions configured to instruct the processor to:
represent the traffic signal device as a raster image in which:
each traffic signal element of the traffic signal device is represented by a mask corresponding to a location of the traffic signal element on the traffic signal device,
each mask is represented in only one of a plurality of channels in the raster image, and
each channel in the raster image is a color channel that indicates a color, feature or both of the traffic signal element represented in the mask;
provide, using the transceiver, the raster image as an input to a neural network to classify a state of each traffic signal element; and
receive, from the neural network, using the transceiver, a classified raster image, in which the classified raster image includes a plurality of masks, each of which represents the state of one of the traffic signal elements.
13. The system of claim 12 , wherein the program instructions to represent the traffic signal device as a raster image comprise instructions to use map data to directly generate the raster image.
14. The system of claim 12 , wherein the program instructions to represent the traffic signal device as a raster image comprise instructions to generate a digital image from map data and convert the digital image to a raster image.
15. The system of claim 12 , wherein the plurality of color channels in the raster image correspond to a plurality of different colors.
16. The system of claim 12 , wherein the color, feature, or both of each of the one or more traffic signal elements represented in the color channels comprise one or more of the following:
a green light;
a yellow light;
a red light;
a circular light;
a left arrow light;
a right arrow light;
a forward arrow light;
a light having an arrow in any direction;
a flashing green light;
a flashing yellow light;
a flashing red light;
a U-turn light;
a bicycle light; or
an X-light.
17. The system of claim 12 , wherein each mask corresponds to a discrete region of pixels in the raster image.
18. The system of claim 12 , wherein the processor is further configured to:
identify a face of the traffic signal device,
wherein each traffic signal element is located within the face of the traffic signal device.
19. A system for representing a traffic signal device, the system comprising:
a processor;
a computer vision system configured to receive a digital image of a traffic signal device that includes one or more traffic signal elements; and
a non-transitory computer-readable storage medium comprising one or more programming instructions that, when executed, cause the processor to:
represent the traffic signal device as a raster image, wherein:
each traffic signal element of the traffic signal device is represented by a mask corresponding to a location of the traffic signal element on the traffic signal device,
each mask is represented in only one of a plurality of in a channels in the raster image, and
each channel in the raster image is a color channel that indicates a color, feature or both of the traffic signal element represented in the mask;
provide the raster image as an input to a neural network to classify a state of each of the one or more traffic signal elements; and
receive, from the neural network, a classified raster image, in which the classified raster image includes a plurality of masks, wherein each mask represents the state of one of the one or more traffic signal elements.
20. The system of claim 19 , wherein the color, feature, or both of each of the one or more traffic signal elements represented in the color channels comprise one or more of the following:
a green light;
a yellow light;
a red light;
a circular light;
a left arrow light;
a right arrow light;
a forward arrow light;
a light having an arrow in any direction;
a flashing green light;
a flashing yellow light;
a flashing red light;
a U-turn light;
a bicycle light; or
an X-light.