Image processing apparatus and image processing method including area setting and perspective conversion
An image processing apparatus and method can combine images, while maintaining continuity without causing discomfort, that capture the surrounding area of an automobile. An image processing apparatus includes an image acquisition unit that acquires first and second images respectively capturing surrounding areas of an automobile including first and second areas, an outline detector that performs outline detection on the first and second images and generates first and second outlines respectively, a determiner that determines whether the first and second outlines each include an outline of the same object, and a area selector that performs area setting and perspective conversion on the first or second image when the first and second outlines are determined to include an outline of the same object, so that in a combined image generated by combining at least the first and second images, the first and second outlines are continuous at the same object.
1. An image processing apparatus comprising:
one or more processors configured to implement:
an image acquisition unit configured to acquire a first image capturing a surrounding area of an automobile including a first area and a second image capturing a surrounding area of the automobile including a second area adjacent to the first area;
an outline detector configured to perform outline detection on the first image to generate a first outline of an object within the first image and perform outline detection on the second image to generate a second outline of an object within the second image, wherein at least one of the object within the first image and the object within the second image extends across a border between the first image and the second image;
a sameness determiner configured to determine, based on the first outline and the second outline, whether the object within the first image and the object within the second image is the same object; and
a combined area selector configured to determine, when the object within the first image and the object within the second image have been determined to be the same object, whether the first outline and the second outline are continuous or discontinuous across the border between the first image and the second image,
wherein, when the combined area selector determines that the first outline and the second outline are discontinuous across the border between the first image and the second image, the combined area selector performs area setting and perspective conversion on the first image or the second image to correct a discontinuity between the first outline and the second outline, and
wherein the combined area selector is further configured to combine at least the first image and the second image to generate a combined image, wherein, in the combined image, the first outline and the second outline are continuous across the border between the first image and the second image.
2. The image processing apparatus of claim 1 , wherein the object within the first image or the object within the second image is any one selected from the group consisting of a display on a road surface, a guardrail, a side ditch, a road shoulder, and a delineator.
3. The image processing apparatus of claim 1 , wherein the object within the first image or the object within the second image is a graphic drawn on a road surface by the automobile irradiating light.
4. An image processing apparatus comprising:
one or more processors configured to implement:
an image acquisition unit configured to acquire a first image capturing a surrounding area of an automobile including a first area and a second image capturing a surrounding area of the automobile including a second area adjacent to the first area;
an outline detector configured to perform outline detection on the first image to generate a first outline of an object within the first image and perform outline detection on the second image to generate a second outline of an object within the second image; and
a combined area selector configured to determine, based on the first outline and the second outline and movement of the automobile, whether the object within the first image and the object within the second image is the same object,
wherein, when the object within the first image and the object within the second image have been determined to be the same object, the combined area selector determines a position and an orientation of the object within the second image based on the first image and the movement of the automobile, performs area setting and perspective conversion on the second image, and combines at least the first image and the second image, in which area setting and perspective correction has been performed, to generate a combined image, and
wherein, in the combined image, the position and orientation of the object within the second image matches a position and an orientation of the object within the first image and corresponds to the movement of the automobile.
5. The image processing apparatus of claim 4 , wherein the combined area selector uses a movement direction and a speed of the automobile.
6. The image processing apparatus of claim 5 , wherein the combined area selector performs the area setting and the perspective conversion on the second image when the automobile is traveling straight.
7. The image processing apparatus of claim 4 , wherein the object within the first image or the object within the second image is any one selected from the group consisting of a display on a road surface, a guardrail, a side ditch, a road shoulder, and a delineator.
8. An image processing method comprising:
acquiring a first image capturing a surrounding area of an automobile including a first area and a second image capturing a surrounding area of the automobile including a second area adjacent to the first area;
performing outline detection on the first image to generate a first outline of an object within the first image and performing outline detection to generate a second outline of an object within the second image, wherein at least one of the object within the first image and the object within the second image extends across a border between the first image and the second image;
sameness determining, based on the first outline and the second outline, whether the object within the first image and the object within the second image is the same object;
determining, when the object within the first image and the object within the second image have been determined to be the same object, whether the first outline and the second outline are continuous or discontinuous across the border between the first image and the second image;
when the first outline and the second outline are determined to be discontinuous across the border between the first image and the second image, performing area setting and perspective conversion on the first image or the second image to correct a discontinuity between the first outline and the second outline when the first outline and the second outline; and
generating a combined image by combining at least the first image and the second image, wherein, in the combined image, the first outline and the second outline are continuous across the border between the first image and the second image.
9. An image processing method comprising:
acquiring a first image capturing a surrounding area of an automobile including a first area and a second image capturing a surrounding area of the automobile including a second area adjacent to the first area;
performing outline detection on the first image to generate a first outline of an object within the first image and performing outline detection on the second image to generate a second outline of an object within the second image;
determining, based on the first outline and the second outline and movement of the automobile, whether the object within the first image and the object within the second image is the same object; and
when the object within the first image and the object within the second image have been determined to be the same object, determining a position and an orientation of the object within the second image based on the first image and the movement of the automobile, performing area setting and perspective conversion on the second image, and combining at least the first image and the second image, in which area setting and perspective correction has been performed, to generate a combined image,
wherein, in the combined image, the position and orientation of the object within the second image matches a position and an orientation of the object within the first image and corresponds to the movement of the automobile.