Object detection apparatus, object detection method, and computer readable recording medium

1. An object detection apparatus for detecting an object in a fish-eye image, the apparatus comprising:

a normalized image acquisition unit configured to acquire a normalized image obtained by normalizing the fish-eye image in which the object appears;

a position detecting unit configured to detect position coordinates of the object in the normalized image; and

a determination unit configured to determine a positional relationship between the object and the apparatus, using the position coordinates of the object in the normalized image,

wherein the normalized image acquisition unit comprises:

a fish-eye image acquisition unit configured to acquire the fish-eye image;

a viewpoint compensation vector acquisition unit configured to acquire a viewpoint compensation vector for transforming the fish-eye image to an image obtained by shooting the object from a direction parallel to a ground plane of the object, based on a roll angle of an imaging device around an optical axis thereof and a pitch angle of the optical axis, with a plane parallel to the ground plane of the object serving as a reference; and

an image generating unit configured to set, for the fish-eye image transformed using the viewpoint compensation vector, a plurality of viewpoints parallel to the ground plane along a horizontal direction of the fish-eye image, perform, for each of the set viewpoints, distortion correction using perspective projection approximation for the transformed fish-eye image, based on a coordinate system that includes a line of sight from the viewpoint as an axis, and generate the normalized image using image elements in a vertical direction that are extracted from each of the corrected fish-eye images.

2. The object detection apparatus according to claim 1 ,

wherein the object detection apparatus is mounted in a vehicle, and

the determination unit transforms the position coordinates of the object in the normalized image to coordinates in a coordinate system that provides position coordinates with a distance of the object to the vehicle and a tilt angle of the object relative to an optical axis of the imaging device, and determines a positional relationship between the object and the vehicle based on a time-series change in the transformed position coordinates of the object.

3. The object detection apparatus according to claim 2 ,

wherein the determination unit obtains a rate of change in the distance of the object to the vehicle and a rate of change in the tilt angle of the object, based on the transformed position coordinates of the object, and determines whether the object is in a state of moving around toward the vehicle, a state of crossing in front of the vehicle, or a state of approaching the vehicle, based on the obtained rate of change in the distance of the object to the vehicle and the obtained rate of change in the tilt angle of the object.

4. The object detection apparatus according to claim 2 ,

wherein the determination unit transforms, using a reference table that is created in advance, the position coordinates of the object in the normalized image to coordinates in a coordinate system that provides position coordinates with a distance of the object to the vehicle and the tilt angle of the object relative to the optical axis of the imaging device, and

coordinates in a vertical direction and a horizontal direction in the normalized image, and a distance and a tilt angle that correspond to the coordinates in the vertical direction and the horizontal direction in the normalized image are registered in association with each other in the reference table.

5. An object detection method for detecting an object in a fish-eye image, the method comprising:

(a) acquiring a normalized image obtained by normalizing the fish-eye image in which the object appears;

(b) detecting position coordinates of the object in the normalized image; and

(c) determining a positional relationship with the object, using the position coordinates of the object in the normalized image,

wherein the acquiring includes:

(a1) acquiring the fish-eye image;

(a2) acquiring a viewpoint compensation vector for transforming the fish-eye image to an image obtained by shooting the object from a direction parallel to a ground plane of the object, based on a roll angle of an imaging device around an optical axis thereof and a pitch angle of the optical axis, with a plane parallel to the ground plane of the object serving as a reference; and

(a3) setting, for the fish-eye image transformed using the viewpoint compensation vector, a plurality of viewpoints parallel to the ground plane along a horizontal direction of the fish-eye image, performing, for each of the set viewpoints, distortion correction using perspective projection approximation for the transformed fish-eye image, based on a coordinate system that includes a line of sight from the viewpoint as an axis, and generating the normalized image using image elements in a vertical direction that are extracted from each of the corrected fish-eye images.

6. The object detection method according to claim 5 ,

wherein the object detection method is performed in a vehicle, and

in the (c), the position coordinates of the object in the normalized image are transformed to coordinates in a coordinate system that provides position coordinates with a distance of the object to the vehicle and a tilt angle of the object relative to an optical axis of the imaging device, and a positional relationship between the object and the vehicle is determined based on a time-series change in the transformed position coordinates of the object.

7. The object detection method claim 6 ,

wherein, in the (c), a rate of change in the distance of the object to the vehicle and a rate of change in the tilt angle of the object are obtained based on the transformed position coordinates of the object, and

it is determined whether the object is in a state of moving around toward the vehicle, a state of crossing in front of the vehicle, or a state of approaching the vehicle, based on the obtained rate of change in the distance of the object to the vehicle and the obtained rate of change in the tilt angle of the object.

8. The object detection method according to claim 6 ,

wherein, in the (c), the position coordinates of the object in the normalized image are transformed, using a reference table that is created in advance, to coordinates in a coordinate system that provides position coordinates with a distance of the object to the vehicle and the tilt angle of the object relative to the optical axis of the imaging device, and

coordinates in a vertical direction and a horizontal direction in the normalized image, and a distance and a tilt angle that correspond to the coordinates in the vertical direction and the horizontal direction in the normalized image are registered in association with each other in the reference table.

9. A non-transitory computer readable recording medium that includes a program recorded thereon, the program being for detecting an object in a fish-eye image using a computer and including instructions that cause the computer to carry out:

(a) a step of acquiring a normalized image obtained by normalizing the fish-eye image in which the object appears;

(b) a step of detecting position coordinates of the object in the normalized image; and

(c) a step of determining a positional relationship with the object, using the position coordinates of the object in the normalized image,

wherein the step of acquiring includes:

(a1) a step of acquiring the fish-eye image;

(a2) a step of acquiring a viewpoint compensation vector for transforming the fish-eye image to an image obtained by shooting the object from a direction parallel to a ground plane of the object, based on a roll angle of an imaging device around an optical axis thereof and a pitch angle of the optical axis, with a plane parallel to the ground plane of the object serving as a reference; and

(a3) a step of setting, for the fish-eye image transformed using the viewpoint compensation vector, a plurality of viewpoints parallel to the ground plane along a horizontal direction of the fish-eye image, performing, for each of the set viewpoints, distortion correction using perspective projection approximation for the transformed fish-eye image, based on a coordinate system that includes a line of sight from the viewpoint as an axis, and generating the normalized image using image elements in a vertical direction that are extracted from each of the corrected fish-eye images.

10. The non-transitory computer readable recording medium according to claim 9 ,

wherein the computer is mounted in a vehicle, and

in the (c) step, the position coordinates of the object in the normalized image are transformed to coordinates in a coordinate system that provides position coordinates with a distance of the object to the vehicle and a tilt angle of the object relative to an optical axis of the imaging device, and a positional relationship between the object and the vehicle is determined based on a time-series change in the transformed position coordinates of the object.

11. The non-transitory computer readable recording medium according to claim 10 ,

wherein, in the (c) step, a rate of change in the distance of the object to the vehicle and a rate of change in the tilt angle of the object are obtained based on the transformed position coordinates of the object, and

it is determined whether the object is in a state of moving around toward the vehicle, a state of crossing in front of the vehicle, or a state of approaching the vehicle, based on the obtained rate of change in the distance of the object to the vehicle and the obtained rate of change in the tilt angle of the object.

12. The computer readable recording medium according to claim 10 ,

wherein, in the (c) step, the position coordinates of the object in the normalized image are transformed, using a reference table that is created in advance, to coordinates in a coordinate system that provides position coordinates with a distance of the object to the vehicle and the tilt angle of the object relative to the optical axis of the imaging device, and

coordinates in a vertical direction and a horizontal direction in the normalized image, and a distance and a tilt angle that correspond to the coordinates in the vertical direction and the horizontal direction in the normalized image are registered in association with each other in the reference table.