Image processing device, image processing method, and program
Captured images generated by repeated image capture over time by an image capture device provided on a moving body are acquired, and for each pixel in each of said captured images, a probability indicating to which of multiple different area classes, which have been designated regarding multiple targets appearing in the captured images, the pixel belongs is calculated. Among divisional areas obtained by multiply dividing the captured images in a horizontal direction, a divisional area in which a clarity of a prescribed target among the multiple targets is high, where the probability of belonging to the area class of the prescribed target is equal to or higher than a threshold value, is identified. A clear image is generated by combining the divisional areas in which a display clarity regarding the prescribed target in the captured images is high.
1 . An image processing device comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to:
acquire a plurality of captured images generated by repeated image capture over time by an image capture device provided on a moving body, and calculating, for each pixel in each of the plurality of captured images, a probability that the pixel belongs to each of multiple different area classes indicating multiple targets appearing in the plurality of captured images;
divide each of the plurality of captured images into divisional areas in a horizontal direction;
identify, among the divisional areas of each of the plurality of captured images, divisional areas, based on a range of pixels, in which the probability of belonging to an area class indicating a prescribed target among the multiple targets is equal to or higher than a threshold value; and
generate a clear image regarding the prescribed target by combining the identified divisional areas in the plurality of captured images.
2 . The image processing device according to claim 1 , wherein the at least one processor is configured to execute the instructions to:
generate the clear image by combining the identified divisional areas regarding the prescribed target among the divisional areas in bird's-eye images generated based on the plurality of captured images, and by returning a bird's-eye image generated by the combination to a viewpoint of the image capture device.
3 . The image processing device according to claim 1 , wherein
the at least one processor is further configured to execute the instructions to acquire movement amounts of the moving body between image capture timings of the plurality of captured images, and
the at least one processor is configured to execute the instructions to generate the clear image by combining the divisional areas, in which each of the plurality of captured images has been divided according to ratios between the acquired movement amounts.
4 . The image processing device according to claim 1 , wherein
the at least one processor is further configured to execute the instructions to calculate positions of moving bodies appearing in the clear image.
5 . The image processing device according to claim 1 , wherein:
the moving body is a vehicle;
the plurality of captured images are images capturing, in a direction of advancement, a road on which the vehicle is traveling; and
the prescribed target is a line painted on a road surface of the road.
6 . The image processing device according to claim 1 , wherein the at least one processor is configured to execute the instructions to:
acquire a number of divisions based on a frame rate of the plurality of captured images or a speed of the moving body, and identify the divisional areas obtained by dividing each of the plurality of captured images according to the acquired number of divisions.
7 . An image processing method that comprises:
acquiring a plurality of captured images generated by repeated image capture over time by an image capture device provided on a moving body, and calculating, for each pixel in each of the plurality of captured images, a probability that the pixel belongs to each of multiple different area classes indicating multiple targets appearing in the plurality of captured images;
dividing each of the plurality of captured images into divisional areas in a horizontal direction:
identifying, among the divisional areas of each of the plurality of captured images, divisional areas, based on a range of pixels, in which the probability of belonging to an area class indicating a prescribed target among the multiple targets is equal to or higher than a threshold value; and
generating a clear image regarding the prescribed target by combining the identified divisional areas in the plurality of captured images.
8 . A non-transitory computer-readable storage medium storing program that makes a computer in an image processing device to execute processes, the processes comprising:
acquiring a plurality of captured images generated by repeated image capture over time by an image capture device provided on a moving body, and calculating, for each pixel in each of the plurality of captured images, a probability that the pixel belongs to each of multiple different area classes indicating multiple targets appearing in the plurality of captured images;
dividing each of the plurality of captured images into divisional areas in a horizontal direction;
identifying, among the divisional areas of each of the plurality of captured images, divisional areas based on a range of pixels in which a probability of belonging to an area class indicating a prescribed target among the multiple targets is equal to or higher than a threshold value; and
generating a clear image regarding the prescribed target by combining the identified divisional areas in the plurality of captured images.
9 . The image processing device according to claim 1 , wherein
in a case where a predetermined amount of the divisional areas cannot be identified, acquiring, dividing and identifying are performed based on the plurality of captured images captured at respective locations obtained by slightly shifting a location in a direction of advancement of the moving body by a predetermined distance.
10 . The image processing device according to claim 1 , wherein the at least one processor is configured to execute the instructions to:
acquire the plurality of captured images by repeatedly capturing an image in a direction of advancement over time by the image capture device provided on the moving body that is traveling.
11 . The image processing device according to claim 1 , wherein the at least one processor is configured to execute the instructions to:
identify the divisional areas based on a size or shape of the range of pixels in which the probability of belonging to the area class indicating the prescribed target is equal to or higher than the threshold value.
12 . The image processing device according to claim 1 , wherein the at least one processor is configured to execute the instructions to:
generate the clear image by combining the identified divisional areas, divided in the horizontal direction, in a vertical direction.
13 . The image processing device according to claim 1 , wherein the plurality of captured images are divided only in the horizontal direction.
14 . The image processing device according to claim 13 , wherein the at least one processor is configured to execute the instructions to:
acquire the plurality of captured images by repeatedly capturing an image in a direction of advancement over time by the image capture device provided on the moving body that is traveling, and
wherein the horizontal direction is perpendicular to the direction of advancement.