Image processing apparatus

1 . An in-vehicle apparatus mounted in a vehicle and comprising:

a camera positioned on the vehicle to capture an image of a scene in front of the vehicle, and

a controller including a memory and a processor, the processor configured to identify a light emitting state of a traffic light having a green light, a yellow light and a red light from the image captured by the camera, the light emitting state being one of green, yellow and red, the processor configured to:

(i) perform image recognition of the image captured by the camera to generate a traffic light image of the traffic light in the camera image;

(ii) detect, from the traffic light image, each of green pixels, yellow pixels, and red pixels of the green, yellow and red lights of the traffic light based on predetermined threshold ranges for each of green, yellow and red color components;

(iii) identify a light emitting region and a plurality of non-light emitting regions from the green pixels, the yellow pixels and the red pixels that have been detected by scanning the traffic light image with a detection filter defining (1) a light-emitting region and (2) a plurality of non-light-emitting regions at predetermined relative positions corresponding to an arrangement of the green, yellow and red lights in the traffic light;

(iv) calculate an average luminance and an average saturation of the light-emitting region and of the plurality of non-light-emitting regions;

(v) determine the light-emitting region corresponds to a light, of the green, yellow and red lights of the traffic light, that is emitting light when:

differences between the average luminance of the light-emitting region and each of the non-light-emitting regions exceed a first threshold;

differences between the average luminance of the light-emitting region and each of the non-light-emitting regions exceed a second threshold;

a number of pixels in the light-emitting region satisfying a color condition exceeds a predetermined count threshold; and

an average color value of each of the non-light-emitting regions falls within a predetermined range; and

(vi) determine the light emitting state of the traffic light based on a color corresponding to the light of the light emitting region.

2 . The in-vehicle apparatus according to claim 1 , wherein

the processor is configured to identify positions of the non-light emitting regions depending on the green pixels, the yellow pixels, and the red pixels in the light-emitting region that has been identified.

3 . The in-vehicle apparatus according to claim 1 , wherein

the processor is configured to perform a color space adjustment on the image captured by the camera in a YUV format, and

the differences between the average luminance and the differences between the average saturation of the light emitting region and each of the non-light emitting regions are determined after the color space adjustment has been performed.

4 . The in-vehicle apparatus according to claim 1 , wherein

the light emitting state is determined based on information of a plurality of frames in the image captured by the camera.

5 . An image processing method that determines a light emitting state of a traffic light having a green light, a yellow light and a red light from an image captured by a camera of an in-vehicle apparatus that is mounted in a vehicle, the image captured by the camera being of a scene in front of the vehicle, the method being executed by a processor of the in-vehicle apparatus and comprising the steps of:

(i) preforming image recognition of the image captured by the camera to generate a traffic light image of the traffic light in the camera image;

(ii) detecting, from the traffic light image, each of green pixels, yellow pixels, and red pixels of the green, yellow and red lights of the traffic light based on predetermined threshold ranges for each of green, yellow and red color components;

(iii) identifying a light emitting region and a plurality of non-light emitting regions from the green pixels, the yellow pixels and the red pixels that have been detected by scanning the traffic light image with a detection filter defining (1) a light-emitting region and (2) a plurality of non-light-emitting regions at a predetermined relative positions corresponding to an arrangement of the green, yellow and red lights in the traffic light;

(iv) calculating an average luminance and an average saturation of the light-emitting region and of the plurality of non-light-emitting regions;

(v) determining that the light-emitting region corresponds to a light, of the green, yellow and red lights of the traffic light, that is emitting light when:

differences between the average luminance of the light-emitting region and each of the non-light-emitting regions exceed a first threshold;

differences between the average luminance of the light-emitting region and each of the non-light-emitting regions exceed a second threshold;

a number of pixels in the light-emitting region satisfying a color condition exceeds a predetermined count threshold; and

an average color value of each of the non-light-emitting regions falls within a predetermined range; and

(vi) determining the light emitting stat of the traffic light based on a color corresponding to the light of the light emitting region.

6 . A non-transitory computer-readable recording medium having stored therein a program that causes a computer of an in-vehicle apparatus mounted in a vehicle to execute a process that determines a light emitting state of a traffic light having a green light, a yellow light and a red light from an image captured by a camera of the in-vehicle apparatus, the image captured by the camera being of a scene in front of the vehicle, the process comprising:

(i) performing image recognition of the image captured by the camera to generate a traffic light image of the traffic light in the camera image;

(ii) detecting, from the traffic light image, each of green pixels, yellow pixels, and red pixels of the green, yellow and red lights of the traffic light based on predetermined threshold ranges for each of green, yellow and red color components;

(iii) identifying a light emitting region and a plurality of non-light emitting regions from the green pixels, the yellow pixels and the red pixels that have been detected by scanning the traffic light image with a detection filter defining (1) a light-emitting region and (2) a plurality of non-light-emitting regions at a predetermined relative positions corresponding to an arrangement of the green, yellow and red lights in the traffic light;

(iv) calculating an average luminance and an average saturation of the light-emitting region and of the plurality of non-light-emitting regions;

(v) determining that the light-emitting region corresponds to a light, of the green, yellow and red lights of the traffic light, that is emitting light when:

differences between the average luminance of the light-emitting region and each of the non-light-emitting regions exceed a first threshold;

differences between the average luminance of the light-emitting region and each of the non-light-emitting regions exceed a second threshold;

a number of pixels in the light-emitting region satisfying a color condition exceeds a predetermined count threshold; and

an average color value of each of the non-light-emitting regions falls within a predetermined range; and

(vi) determining the light emitting stat of the traffic light based on a color corresponding to the light of the light emitting region.