Object detection apparatus, object detection method, and computer-readable recording medium

1. An object detection apparatus, comprising:

a processor;

a memory device storing program code that the processor is to execute to function as:

a fish-eye image acquisition unit configured to acquire a time series fish-eye image output from a fish-eye camera;

a horizontal panorama image generation unit configured to, for each frame of a plurality of frames included in the time series fish-eye image, perform conversion to a horizontal panorama image in which a vertical direction in a real space is expressed in a perpendicular direction of the frame, and an azimuth is expressed equiangularly in a horizontal direction of the frame;

an edge pair extraction unit configured to extract, for each frame, a pair of edges in the perpendicular direction from the horizontal panorama image;

a change rate extraction unit configured to extract, between each adjacent pair of the frames, a change rate of an inter-edge distance between the pair of edges extracted for each frame of the adjacent pair;

a lower end region extraction unit configured to extract, for each frame, a region that corresponds to a lower end of an object predicted to be providing the pair of edges;

a distance change rate extraction unit configured to calculate, for each frame, a distance from the object to the fish-eye camera based on A position of the region that corresponds to the lower end of the object in the horizontal panorama image, and extract, between each adjacent pair of the frames, a change rate of the distance from the object to the fish-eye camera; and

an object detection unit configured to determine whether or not the object exists based on the change rate of the inter-edge distance and the change rate of the distance from the object to the fish-eye camera that were extracted.

2. The object detection apparatus according to claim 1 ,

wherein the horizontal panorama image generation unit,

based on a roll angle around an optical axis of the fish-eye camera and a pitch angle of the optical axis with respect to a plane parallel to a ground plane of the object, generates a viewpoint compensation vector that converts one of the frames to an image obtained by capturing an image of the object from a direction parallel to the ground plane,

with respect to the one of the frames that was converted using the viewpoint compensation vector, sets a plurality of viewpoints parallel to the ground plane in the horizontal direction of the one of the frames,

for each of the viewpoints that was set, on the one of the frames after the conversion, based on a coordinate system that includes a sight line from the viewpoint as an axis, performs distortion correction by perspective projection approximation, and

using an image element in the perpendicular direction extracted from each of the frames after correction, generates one new frame, and adopts the one new frame as the horizontal panorama image.

3. The object detection apparatus according to claim 1 ,

wherein the object detection unit determines that the object exists when there is an inverse relationship between the change rate of the inter-edge distance and the change rate of the distance from the object to the fish-eye camera.

4. The object detection apparatus according to claim 1 ,

wherein the fish-eye camera is disposed such that, in a vehicle, an optical axis direction is a direction inclined downward in a vertical direction from a horizontal plane.

5. An object detection method, comprising:

acquiring a time series fish-eye image output from a fish-eye camera;

for each frame of a plurality of frames included in the time series fish-eye image, performing conversion to a horizontal panorama image in which a vertical direction in a real space is expressed in a perpendicular direction of the frame, and an azimuth is expressed equiangularly in a horizontal direction of the frame:

for each frame, a pair of edges in the perpendicular direction from the horizontal panorama image:

extracting, between each adjacent pair of the frames, a change rate of an inter-edge distance between the pair of edges extracted for each frame of the adjacent pair;

for each frame, a region that corresponds to a lower end of an object predicted to be providing the pair of edges;

for each frame, a distance from the object to the fish-eye camera based on a position of the region that corresponds to the lower end of the object in the horizontal panorama image, and extracting, between each adjacent pair of the frames, a change rate of the distance from the object to the fish-eye camera; and

determining whether or not the object exists based on the change rate of the inter-edge distance and the change rate of the distance from the object to the fish-eye camera that were extracted.

6. The object detection method according to claim 5 ,

wherein in the performing of the conversion,

based on a roll angle around an optical axis of the fish-eye camera and a pitch angle of the optical axis with respect to a plane parallel to a ground plane of the object, a viewpoint compensation vector that converts one of the frames to an image obtained by capturing an image of the object from a direction parallel to the ground plane is generated,

with respect to the one of the frames that was converted using the viewpoint compensation vector, a plurality of viewpoints parallel to the ground plane in the horizontal direction of the frame are set,

for each of the viewpoints that was set, on the one of the frames after the conversion, based on a coordinate system that includes a sight line from the viewpoint as an axis, distortion correction by perspective projection approximation is performed, and

using an image element in the perpendicular direction extracted from each of the frames after correction, one new frame is generated, and this is adopted as the horizontal panorama image.

7. The object detection method according to claim 5 ,

wherein in the determining, the object is determined to exist when there is an inverse relationship between the change rate of the inter-edge distance and the change rate of the distance from the object to the fish-eye camera.

8. The object detection method according to claim 5 ,

wherein the fish-eye camera is disposed such that, in a vehicle, an optical axis direction is a direction inclined downward in a vertical direction from a horizontal plane.

9. A non-transitory computer-readable recording medium that includes a program recorded thereon, the program including instructions that cause a computer to carry out:

acquiring a time series fish-eye image output from a fish-eye camera:

for each frame of a plurality of frames included in the time series fish-eye image, performing conversion to a horizontal panorama image in which a vertical direction in a real space is expressed in a perpendicular direction of the frame, and an azimuth is expressed equiangularly in a horizontal direction of the frame;

extracting, for each frame, a pair of edges in the perpendicular direction from the horizontal panorama image;

extracting, between each adjacent pair of the frames, a change rate of an inter-edge distance between the pair of edges extracted for each frame of the adjacent pair;

extracting, for each frame, a region that corresponds to a lower end of an object predicted to be providing the pair of edges;

calculating, for each frame, a distance from the object to the fish-eye camera based on a position of the region that corresponds to the lower end of the object in the horizontal panorama image, and extracting, between each adjacent air of the frames, a change rate of the distance from the object to the fish-eye camera; and

determining whether or not the object exists based on the change rate of the inter-edge distance and the change rate of the distance from the object to the fish-eye camera that were extracted.

10. The non-transitory computer-readable recording medium according to claim 9 ,

wherein in the performing of the conversion,

based on a roll angle around an optical axis of the fish-eye camera and a pitch angle of the optical axis with respect to a plane parallel to a ground plane of the object, a viewpoint compensation vector that converts the frame to an image obtained by capturing an image of the object from a direction parallel to the ground plane is generated,

with respect to one of the frames that was converted using the viewpoint compensation vector, a plurality of viewpoints parallel to the ground plane in the horizontal direction of the frame are set,

for each of the viewpoints that was set, on the one of the frames after the conversion, based on a coordinate system that includes a sight line from the viewpoint as an axis, distortion correction by perspective projection approximation is performed, and

using an image element in the perpendicular direction extracted from each of the frames after correction, one new frame is generated, and this is adopted as the horizontal panorama image.

11. The non-transitory computer-readable recording medium according to claim 9 ,

wherein in the determining, the object is determined to exist when there is an inverse relationship between the change rate of the inter-edge distance and the change rate of the distance from the object to the fish-eye camera.

12. The non-transitory computer-readable recording medium according to claim 9 ,

wherein the fish-eye camera is disposed such that, in a vehicle, an optical axis direction is a direction inclined downward in a vertical direction from a horizontal plane.