Lane detection and distance estimation using single-view geometry
Disclosed are methods, devices, and computer-readable media for detecting lanes and objects in image frames of a monocular camera. In one embodiment, a method is disclosed comprising receiving a sample set of image frames; detecting a plurality of markers in the sample set of image frames using a convolutional neural network (CNN); fitting lines based on the plurality of markers; detecting a plurality of vanishing points based on the lines; identifying a best fitting horizon for the sample set of image frames via a RANSAC algorithm; computing an inverse perspective mapping (IPM) based on the best fitting horizon; and computing a lane width estimate based on the sample set of image frames using the IPM in a rectified view and the parallel line fitting.
1. A method comprising:
receiving a sample set of image frames;
detecting a plurality of markers in the sample set of image frames using a convolutional neural network (CNN);
fitting lines based on the plurality of markers;
detecting a plurality of vanishing points based on the lines;
performing a random sample consensus (RANSAC) algorithm on the plurality of vanishing points to identify a best fitting horizon for the sample set of image frames;
computing an inverse perspective mapping (IPM) based on the best fitting horizon; and
computing a lane width estimate based on the sample set of image frames using the IPM in a rectified view and the line fitting.
2. The method of claim 1 , wherein detecting a plurality of markers comprises detecting, using a convolutional neural network (CNN), a plurality of markers situated along a lane boundary of a roadway included in the sample set of image frames.
3. The method of claim 1 , wherein computing the IPM comprises computing a rectification homography based on an intrinsic matrix of a camera used to capture the sample set of image frames and a rotation matrix, the rotation matrix calculated based on the best fitting horizon and a road plane normal.
4. The method of claim 1 , further comprising rectifying the sample set of image frames using the IPM to generate a rectified view.
5. The method of claim 1 , further comprising computing a height of a camera that captured the sample set of image frames based on a known lane width in real world.
6. The method of claim 1 , further comprising receiving a second set of frames and detecting a plurality of lanes in the second set of frames using the IPM.
7. The method of claim 6 , wherein detecting a plurality of lane lines in the second set of frames using the IPM comprises:
rectifying markers detected in the second set of frames using the IPM; and
fitting a plurality of lines to the rectified markers.
8. The method of claim 7 , further comprising predicting at least one missing lane boundary in the rectified view.
9. The method of claim 1 , further comprising perspectively transforming points associated with the lines using an inverse of the IPM.
10. The method of claim 9 , further comprising:
detecting an object in the sample set of image frames; and
calculating a distance from a camera to the object based on a camera height and a normal plane.
11. A non-transitory computer-readable storage medium for tangibly storing computer program instructions capable of being executed by a computer processor, the computer program instructions defining the steps of:
receiving a sample set of image frames;
detecting a plurality of markers in the sample set of image frames using a convolutional neural network (CNN);
fitting lines based on the plurality of markers;
detecting a plurality of vanishing points based on the lines;
performing a random sample consensus (RANSAC) algorithm on the plurality of vanishing points to identify a best fitting horizon for the sample set of image frames;
computing an inverse perspective mapping (IPM) based on the best fitting horizon; and
computing a lane width estimate based on the sample set of image frames using the IPM in a rectified view and the line fitting.
12. The non-transitory computer-readable storage medium of claim 11 , wherein detecting a plurality of markers comprises detecting, using a convolutional neural network (CNN), a plurality of markers situated along a lane boundary of a roadway included in the sample set of image frames.
13. The non-transitory computer-readable storage medium of claim 11 , wherein computing the IPM comprises computing a rectification homography based on an intrinsic matrix of a camera used to capture the sample set of image frames and a rotation matrix, the rotation matrix calculated based on the best fitting horizon and a road plane normal.
14. The non-transitory computer-readable storage medium of claim 10 , the steps further defining the step of computing a height of a camera that captured the sample set of image frames based on a known lane width.
15. The non-transitory computer-readable storage medium of claim 10 , the steps further defining the step of receiving a second set of frames and detecting a plurality of lanes in the second set of frames using the IPM.
16. The non-transitory computer-readable storage medium of claim 15 , wherein detecting a plurality of lane lines in the second set of frames using the IPM comprises:
rectifying markers detected in the second set of frames using the IPM; and
fitting a plurality of lines to the rectified markers.
17. The non-transitory computer-readable storage medium of claim 16 , the steps further defining the step of predicting at least one missing lane boundary in the rectified view.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the steps further defining the steps of:
detecting an object in the sample set of image frames; and
calculating a distance from a camera to the object based on a camera height and a normal plane.
19. A device comprising:
a processor; and
a storage medium for tangibly storing thereon program logic for execution by the processor for performing steps of:
receiving a sample set of image frames;
detecting a plurality of markers in the sample set of image frames using a convolutional neural network (CNN);
fitting lines based on the plurality of markers;
detecting a plurality of vanishing points based on the lines;
performing a random sample consensus (RANSAC) algorithm on the plurality of vanishing points to identify a best fitting horizon for the sample set of image frames;
computing an inverse perspective mapping (IPM) based on the best fitting horizon; and
computing a lane width estimate based on the sample set of image frames using the IPM in a rectified view and the line fitting.
20. The device of claim 19 , the program logic further defining a step of receiving a second set of frames and detecting a plurality of lane lines in the second set of frames using the IPM.