Image processing apparatus, stereo camera apparatus, vehicle, and image processing method
An image processing apparatus includes an input interface configured to acquire images captured by a camera and a controller configured to detect an edge corresponding to at least one of two or more parallel, straight lines included in the subject of the camera from each of a plurality of frames acquired by the input interface, to select a frame for use by evaluating the detected edge, and to calculate calibration data for the camera on the basis of an approximate straight line corresponding to edge pixels obtained from a plurality of selected frames.
1. An image processing apparatus comprising:
an input interface configured to acquire video including a plurality of frames from a camera; and
a controller configured to detect an edge corresponding to at least one of two or more parallel, straight lines in a subject of the camera from each of the plurality of frames acquired by the input interface, to select a frame for use by evaluating the detected edge, and to calculate calibration data for the video acquired from the camera on the basis of a straight line corresponding to the edge obtained from a plurality of selected frames, wherein
the controller calculates a quadratic or higher order function representing an approximate curve corresponding to the edge, determines whether an absolute value of at least a quadratic coefficient of the quadratic or higher order function is less than a predetermined threshold, and selects the frame for use when determining that the absolute value of the quadratic coefficient is less than a predetermined threshold.
2. The image processing apparatus of claim 1 , wherein the controller detects an edge detected in a region of interest of a frame as the edge corresponding to at least one of two or more parallel, straight lines in the subject.
3. The image processing apparatus of claim 1 , wherein the controller determines whether a number of pixels of the detected edge is greater than or equal to a predetermined threshold and selects the frame for use when determining that the number of pixels of the detected edge is greater than or equal to a predetermined threshold.
4. The image processing apparatus of claim 1 , wherein the controller calculates a least squares error of a straight line corresponding to the edge, determines whether the least squares error is less than a predetermined threshold, and selects the frame for use when determining that the least squares error is less than a predetermined threshold.
5. The image processing apparatus of claim 1 , wherein the video comprises images capturing two or more parallel lane markings on a road surface in three-dimensional (3D) coordinate space, and the edge corresponds to an edge of the lane markings.
6. The image processing apparatus of claim 1 , wherein the controller calculates calibration data for the camera by detecting edges corresponding to at least two lines, calculating an image characteristic amount on the basis of straight lines corresponding to at least two of the edges obtained from the plurality of selected frames, and performing statistical processing on image characteristic amounts of the plurality of frames.
7. The image processing apparatus of claim 6 , wherein the image characteristic amount is a coordinate of a vanishing point obtained from lines of two of the edges.
8. The image processing apparatus of claim 7 , wherein as the statistical processing, the controller generates a histogram of the image characteristic amounts of the plurality of frames and calculates a mode of the image characteristic amounts.
9. The image processing apparatus of claim 6 , wherein as the statistical processing, the controller generates a histogram of the image characteristic amounts of the plurality of frames and calculates a mode of the image characteristic amounts.
10. The image processing apparatus of claim 1 , wherein
the input interface acquires a benchmark image and a reference image captured by a stereo camera, and
the controller calculates calibration data for calibrating the reference image relative to the benchmark image on the basis of a straight line corresponding to an edge obtained from each of the benchmark image and the reference image.
11. A stereo camera apparatus comprising:
a stereo camera; and
an image processing apparatus comprising an input interface configured to acquire video including a plurality of frames from the stereo camera; and a controller configured to detect an edge corresponding to at least one of two or more parallel, straight lines in a subject of the stereo camera from each of the plurality of frames acquired by the input interface, to select a frame for use by evaluating the detected edge, and to calculate calibration data for the video acquired from the stereo camera on the basis of a straight line corresponding to the edge obtained from a plurality of selected frames, wherein
the controller calculates a quadratic or higher order function representing an approximate curve corresponding to the edge, determines whether an absolute value of at least a quadratic coefficient of the quadratic or higher order function is less than a predetermined threshold, and selects the frame for use when determining that the absolute value of the quadratic coefficient is less than a predetermined threshold.
12. A vehicle provided with a stereo camera apparatus, the stereo camera apparatus comprising:
a stereo camera; and
an image processing apparatus comprising an input interface configured to acquire video including a plurality of frames from the stereo camera; and a controller configured to detect an edge corresponding to at least one of two or more parallel, straight lines in a subject of the stereo camera from each of the plurality of frames acquired by the input interface, to select a frame for use by evaluating the detected edge, and to calculate calibration data for the video acquired from the stereo camera on the basis of a straight line corresponding to the edge obtained from a plurality of selected frames, wherein
the controller calculates a quadratic or higher order function representing an approximate curve corresponding to the edge, determines whether an absolute value of at least a quadratic coefficient of the quadratic or higher order function is less than a predetermined threshold, and selects the frame for use when determining that the absolute value of the quadratic coefficient is less than a predetermined threshold.
13. An image processing method comprising:
acquiring, with an input interface, video including a plurality of frames from a camera;
detecting, with a controller, an edge corresponding to at least one of two or more parallel, straight lines in a subject of the camera from each image of the plurality of frames acquired by the input interface;
selecting, with the controller, a frame for use by evaluating the detected edge; and
calculating, with the controller, calibration data for the video acquired from the camera on the basis of a straight line corresponding to the edge obtained from a plurality of selected frames, wherein
the controller calculates a quadratic or higher order function representing an approximate curve corresponding to the edge, determines whether an absolute value of at least a quadratic coefficient of the quadratic or higher order function is less than a predetermined threshold, and selects the frame for use when determining that the absolute value of the quadratic coefficient is less than a predetermined threshold.