Lane marker detection and lane instance recognition

1. A method of estimating lanes, the method comprising:

estimating one or more first lane boundaries of a first camera frame based on a local mask segmentation estimation process;

estimating one or more second lane boundaries of a second camera frame based on a control points estimation process; and

estimating one or more lanes based on the one or more first lane boundaries and the one or more second lane boundaries,

wherein the local mask segmentation estimation process and the control points estimation process are different processes,

wherein inputs to the local mask segmentation estimation process are not based on outputs from the control points estimation process, and

wherein inputs to the control points estimation process are not based on outputs from the local mask segmentation estimation process.

2. The method of claim 1 , wherein at least one first lane boundary and at least one second lane boundary are the same.

3. The method of claim 1 , further comprising:

predicting a first set of one or more lane boundary boxes (LBBs) within the first camera frame prior to estimating the one or more first lane boundaries, the first set including a first LBB; and

predicting a second set of one or more lane boundary boxes (LBBs) within the second camera frame prior to estimating the one or more second lane boundaries, the second set including a second LBB.

4. The method of claim 3 ,

wherein the first and second camera frames are the same, and

wherein the first LBB overlaps, at least in part, with the second LBB.

5. The method of claim 4 , wherein the first and second LBBs are the same.

6. The method of claim 3 ,

wherein the first LBB is less than an entirety of the first camera frame, and/or

wherein the second LBB is less than an entirety of the second camera frame.

7. The method of claim 3 , wherein estimating the one or more first lane boundaries of the first camera frame based on the local mask segmentation estimation process comprises:

estimating locations of one or more lane markers (LMs) within the first LBB based on processing of one or more camera frames prior to the first camera frame; and

sampling for LMs within the first LBB based on the estimated LM locations,

wherein the one or more first lane boundaries comprise one or more lane boundaries corresponding to the one or more LMs sampled within the first LBB.

8. The method of claim 7 , wherein the locations of the one or more LMs are estimated based on one or more camera frames prior to the first camera frame.

9. The method of claim 3 , wherein estimating the one or more second lane boundaries of the second camera frame based on the control points estimation process comprises:

predicting one or more lane boundaries within the second LBB based on processing of one or more camera frames prior to the second camera frame;

predicting a plurality of curve parameters of at least one predicted second lane boundary; and

sampling for lane markers (LMs) within the second LBB based on the plurality of curve parameters,

wherein the one or more second lane boundaries comprise one or more lane boundaries corresponding to the one or more LMs sampled within the second LBB.

10. The method of claim 9 , wherein regression is performed to predict the curve parameters of the at least one predicted lane boundary.

11. The method of claim 3 , wherein estimating the one or more lanes comprises:

combining the one or more first lane boundaries and the one or more second lane boundaries into one or more combined lane boundaries;

updating the one or more combined lane boundaries into one or more updated lane boundaries; and

estimating the one or more lanes based on the updated one or more lane boundaries.

12. The method of claim 11 ,

wherein updating the one or more lane combined lane boundaries comprises rejecting outliers from the one or more combined lane boundaries; and

wherein estimating the one or more lanes based on the updated one or more lane boundaries comprises sampling for lane markers (LMs) based on the updated one or more lane boundaries.

13. The method of claim 12 , wherein rejecting the outliers in the same LBB comprises performing random sample consensus (RANSAC) on the one or more combined lane boundaries.

14. The method of claim 11 ,

wherein the first and second camera frames are the same camera frame and the first and second LBBs are the same LBB,

wherein the same LBB is one of a plurality of LBBs predicted within the same camera frame,

wherein the local mask segmentation estimation process is performed on each LBB of the plurality of LBBs,

wherein the control points estimation process is performed on each LBB of the plurality of LBBs, and

wherein for each LBB, the results of the local mask segmentation estimation process and of the control points estimation process are combined, the lane boundaries of the combined result are updated, and lane estimation based on the updated lane boundaries is performed.

15. An apparatus, comprising:

a memory; and

a processor communicatively coupled to the memory,

wherein the memory and/or the processor are configured to:

estimate one or more first lane boundaries of a first camera frame based on a local mask segmentation estimation process;

estimate one or more second lane boundaries of a second camera frame based on a control points estimation process; and

estimate one or more lanes based on the one or more first lane boundaries and the one or more second lane boundaries,

wherein the local mask segmentation estimation process and the control points estimation process are different processes,

wherein inputs to the local mask segmentation estimation process are not based on outputs from the control points estimation process, and

wherein inputs to the control points estimation process are not based on outputs from the local mask segmentation estimation process.

16. The apparatus of claim 15 , wherein at least one first lane boundary and at least one second lane boundary are the same.

17. The apparatus of claim 15 , wherein the memory and/or the processor are further configured to:

predict a first set of one or more lane boundary boxes (LBBs) within the first camera frame prior to estimating the one or more first lane boundaries, the first set including a first LBB; and

predict a second set of one or more lane boundary boxes (LBBs) within the second camera frame prior to estimating the one or more second lane boundaries, the second set including a second LBB.

18. The apparatus of claim 17 ,

wherein the first and second camera frames are the same, and

wherein the first LBB overlaps, at least in part, with the second LBB.

19. The apparatus of claim 18 , wherein the first and second LBBs are the same.

20. The apparatus of claim 17 ,

wherein the first LBB is less than an entirety of the first camera frame, and/or

wherein the second LBB is less than an entirety of the second camera frame.

21. The apparatus of claim 17 ,

wherein to estimate the one or more first lane boundaries of the first camera frame based on the local mask segmentation estimation process, the memory and/or the processor are further configured to:

estimate locations of one or more lane markers (LMs) within the first LBB based on processing of one or more camera frames prior to the first camera frame; and

sample for LMs within the first LBB based on the estimated LM locations, and

wherein the one or more first lane boundaries comprise one or more lane boundaries corresponding to the one or more LMs sampled within the first LBB.

22. The apparatus of claim 21 , wherein the memory and/or the processor are configured to estimate the locations of the one or more LMs based on one or more camera frames prior to the first camera frame.

23. The apparatus of claim 17 ,

wherein to estimate the one or more second lane boundaries of the second camera frame based on the control points estimation process, the memory and/or the processor are further configured to:

predict one or more lane boundaries within the second LBB based on processing of one or more camera frames prior to the second camera frame;

predict a plurality of curve parameters of at least one predicted second lane boundary; and

sample for lane markers (LMs) within the second LBB based on the plurality of curve parameters, and

wherein the one or more second lane boundaries comprise one or more lane boundaries corresponding to the one or more LMs sampled within the second LBB.

24. The apparatus of claim 23 , wherein the memory and/or the processor are configured to perform regression to predict the curve parameters of the at least one predicted lane boundary.

25. The apparatus of claim 17 , wherein to estimate the one or more lanes, the memory and/or the processor are configured to:

combine the one or more first lane boundaries and the one or more second lane boundaries into one or more combined lane boundaries;

update the one or more combined lane boundaries into one or more updated lane boundaries; and

estimate the one or more lanes based on the updated one or more lane boundaries.

26. The apparatus of claim 25 , wherein the memory and/or the processor are configured to:

update the one or more lane combined lane boundaries by rejecting outliers from the one or more combined lane boundaries; and

estimate the one or more lanes based on the updated one or more lane boundaries by sampling for lane markers (LMs) based on the updated one or more lane boundaries.

27. The apparatus of claim 26 , wherein the memory and/or the processor are configured to reject the outliers in the same LBB by performing random sample consensus (RANSAC) on the one or more combined lane boundaries.

28. The apparatus of claim 25 ,

wherein the first and second camera frames are the same camera frame and the first and second LBBs are the same LBB,

wherein the same LBB is one of a plurality of LBBs predicted within the same camera frame,

wherein the memory and/or the processor are configured to:

perform the local mask segmentation estimation process on each LBB of the plurality of LBBs;

perform the control points estimation process on each LBB of the plurality of LBBs, and

for each LBB, combine the results of the local mask segmentation estimation process and of the control points estimation process, update the lane boundaries of the combined result, and perform lane estimation based on the updated lane boundaries.

29. An apparatus, comprising:

means for estimating one or more first lane boundaries of a first camera frame based on a local mask segmentation estimation process;

means for estimating one or more second lane boundaries of a second camera frame based on a control points estimation process; and

means for estimating one or more lanes based on the one or more first lane boundaries and the one or more second lane boundaries,

wherein the local mask segmentation estimation process and the control points estimation process are different processes, and

wherein inputs to the local mask segmentation estimation process are not based on outputs from the control points estimation process and inputs to the control points estimation process are not based on outputs from the local mask segmentation estimation process.

30. A non-transitory computer-readable medium storing computer-executable instructions for an apparatus, the computer-executable instructions comprising:

one or more instructions causing the apparatus to estimate one or more first lane boundaries of a first camera frame based on a local mask segmentation estimation process;

one or more instructions causing the apparatus to estimate one or more second lane boundaries of a second camera frame based on a control points estimation process; and

one or more instructions causing the apparatus to estimate one or more lanes based on the one or more first lane boundaries and the one or more second lane boundaries,

wherein the local mask segmentation estimation process and the control points estimation process are different processes,

wherein inputs to the local mask segmentation estimation process are not based on outputs from the control points estimation process, and

wherein inputs to the control points estimation process are not based on outputs from the local mask segmentation estimation process.