Method and system for robust curb and bump detection from front or rear monocular cameras

1. A method of detecting a curb comprising:

capturing an image of a path of travel by an image capture device mounted to a vehicle, wherein an optical axis of the image capture device is parallel to a plane of a surface having the curb, and the image capture device has an XYZ camera-center reference system;

applying a feature extraction technique by a processor to the captured image to generate extracted features of the captured image;

applying a classifier to the extracted features to identify a candidate region in the image;

localizing curb edges by the processor in the candidate region of the captured image by extracting edge points;

identifying at least one candidate curb as a function of the extracted edge points;

selecting a pair of parallel curves representing the candidate curb;

determining a range from the image capture device to the candidate curb via the processor;

and

enabling a vehicle application to assist a driver in maneuvering a vehicle utilizing the determined range to the candidate curb;

wherein determining the range includes using the following formula:

D

g

=

h

⁢

w

.

g

u

.

g

where D g is the range, {dot over (w)} g and {dot over (u)} g are the coordinates along respective axes Z and X, in the XYZ camera-center reference system, of a bearing vector b=({dot over (u)},{dot over (v)},{dot over (w)}) of a point on the image, and h is a height of the image capture device above a road surface having the candidate curb.

2. The method of claim 1 wherein applying a feature extraction technique to the captured image includes applying a descriptor for dividing the captured image into cells.

3. The method of claim 2 wherein applying the descriptor includes applying a Histogram of Gradient for dividing the captured image into cells, and wherein a histogram of gradient directions is compiled for each cell as a function of the pixels in each cell.

4. The method of claim 3 wherein the cells have patches defined as small connected regions, and wherein applying a classifier to the extracted features includes applying a binary classifier to classify each of the patches of each cell to identify candidate regions.

5. The method of claim 3 wherein the cells have patches defined as small connected regions, and applying a classifier to the extracted features includes applying a support vector machine to classify each of the patches of each cell to identify candidate regions.

6. The method of claim 1 wherein localized curb edges in the candidate regions includes extracting edge points from the candidate regions.

7. The method of claim 6 wherein a Gaussian filter is applied to smooth the image prior to extracting the edge points.

8. The method of claim 1 wherein identifying candidate curb includes identifying a plurality of candidate curbs that are spaced within a predetermined range of one another.

9. The method of claim 8 wherein the plurality of candidate curbs include five curb lines.

10. The method of claim 8 wherein second order polynomials are used to fit curb lines in the candidate regions.

11. The method of claim 1 wherein the selected pair of parallel curves represent an upper boundary and a lower boundary of the candidate curb.

12. The method of claim 1 wherein determining a range from the image capture device to the candidate curb includes determining a range from a camera plane of the image capture device to the candidate curb.

13. The method of claim 1 further comprising applying temporal smoothing via the processor as a post-processing step to eliminate false positives.

14. The method of claim 13 wherein applying temporal smoothing includes capturing a next image at a next time frame, via the image capture device, and applying a tracking filter to the processed image and the next image via the processor, wherein the temporal smoothing observes redundancies between consecutive image frames.

15. The method of claim 1 wherein enabling a vehicle application includes displaying the image on a display device to the driver and highlighting the identified curb in the display device.

16. The method of claim 1 wherein enabling a vehicle application includes applying the range the candidate curb to an autonomous parking application, the autonomous parking application actuating vehicle devices to park the vehicle.

17. The method of claim 1 wherein enabling a vehicle application includes applying the range to the candidate curb to a collision avoidance application, the collision avoidance application actuating vehicle devices to avoid a collision with the curb.

18. The method of claim 1 wherein enabling a vehicle application includes applying the range to the candidate curb to a clear path detection system, the clear path detection system actuating vehicle devices to maintain the vehicle along the path of travel.

19. The method of claim 1 wherein enabling a vehicle application includes applying the range to the candidate curb to a lane centering application, the lane centering application actuating vehicle devices to center the vehicle within the lane.

20. A method of detecting a curb from a vehicle using an image capture device mounted to the vehicle and having an XYZ camera-center reference system, the method comprising:

capturing an image of a path of travel via the image capture device;

applying a feature extraction technique to the captured image using a processor to thereby generate extracted features of the image;

applying a classifier to the extracted features to identify a candidate region in the image;

localizing curb edges in the identified candidate region of the image by extracting edge points via the processor;

identifying a candidate curb in the identified candidate region as a function of the extracted edge points;

selecting a pair of parallel curves representing the candidate curb;

determining a range from the image capture device to the candidate curb via the processor; and

enabling a vehicle application to assist a driver in maneuvering a vehicle utilizing the determined range of the candidate curb;

wherein determining the range includes using the following formula:

D

g

=

f

⁢

f

u

.

g

where D g is the determined range, {dot over (u)} g is the coordinate along axis X within the XYZ camera-center reference system of a bearing vector b=({dot over (u)},{dot over (v)},{dot over (w)}) of a point on the image, h is a height of the camera above a road surface having the candidate curb, and f is the camera focal length provided by camera intrinsic parameter calibration.

21. The method of claim 1 , further comprising estimating a height (Δh i ) of the candidate curb using the following equation:

Δ

⁢

⁢

h

i

=

D

i

⁢

u

.

g

w

.

g

-

h

;

where D i is the determined range (D g ); and

enabling the vehicle application using the estimated height of the candidate curb.