Implementing autonomous vehicle lane understanding systems using filter-based lane tracking

1 . A method comprising:

obtaining, by a processing device, input data derived from a set of sensors of an autonomous vehicle (AV);

generating, by the processing device, at least one heatmap in a bird's eye view (BEV) representation by applying a set of lane detection classifier heads to a fused BEV feature generated from the input data;

obtaining, by the processing device, a set of polylines in the BEV representation using the at least one heatmap, wherein each polyline of the set of polylines corresponds to a respective track of a first set of tracks for a first frame; and

generating, by the processing device, a second set of tracks for a second frame after the first frame by using a statistical filter based on a set of extrapolated tracks for the second frame and a set of track measurements for the second frame, wherein each track measurement of the set of track measurements corresponds to a respective updated polyline obtained for the second frame.

2 . The method of claim 1 , wherein generating the second set of tracks further comprises:

extrapolating each track of the first set of tracks to generate the set of extrapolated tracks for the second frame; and

obtaining the set of track measurements for the second frame.

3 . The method of claim 1 , wherein generating the second set of tracks further comprises:

associating a track measurement of the set of track measurements to an extrapolated track of the set of extrapolated tracks, wherein the track measurement comprises a set of measured points;

projecting a set of points of the extrapolated track to the track measurement to obtain a set of projected points of the track measurement; and

generating an updated set of points for a track of the second set of tracks based on the set of projected points and the set of measured points.

4 . The method of claim 1 , wherein generating the second set of tracks further comprises creating a new track for a track measurement that is not associated with an extrapolated track of the set of extrapolated tracks.

5 . The method of claim 1 , wherein the input data comprises camera data obtained from at least one camera of the AV and radar data obtained from at least one radar of the AV.

6 . The method of claim 1 , wherein obtaining the input data further comprises generating a dataset with lane labels using auto-labeling.

7 . The method of claim 1 , wherein obtaining the set of polylines further comprises one of:

extracting the set of polylines from the at least one heatmap; or

generating, using a generative model, the set of polylines using the at least one heatmap.

8 . A system comprising:

a memory; and

a processing device communicatively coupled to the memory, the processing device configured to:

obtain input data derived from a set of sensors of an autonomous vehicle (AV);

generate at least one heatmap in a bird's eye view (BEV) representation by applying a set of lane detection classifier heads to a fused BEV feature generated from the input data;

obtain a set of polylines in the BEV representation using the at least one heatmap, wherein each polyline of the set of polylines corresponds to a respective track of a first set of tracks for a first frame; and

generate a second set of tracks for a second frame after the first frame by using a statistical filter based on a set of extrapolated tracks for the second frame and a set of track measurements for the second frame, wherein each track measurement of the set of track measurements corresponds to a respective updated polyline obtained for the second frame.

9 . The system of claim 8 , wherein, to generate the second set of tracks, the processing device is further configured to:

extrapolate each track of the first set of tracks to generate the set of extrapolated tracks for the second frame; and

obtain the set of track measurements for the second frame.

10 . The system of claim 8 , wherein, to generate the second set of tracks, the processing device is further configured to:

associate a track measurement of the set of track measurements to an extrapolated track of the set of extrapolated tracks, wherein the track measurement comprises a set of measured points;

project a set of points of the extrapolated track to the track measurement to obtain a set of projected points of the track measurement; and

generate an updated set of points for a track of the second set of tracks based on the set of projected points and the set of measured points.

11 . The system of claim 8 , wherein, to generate the set of updated track measurements for the second frame, the processing device is further configured to create a new track for a track measurement that is not associated with an extrapolated track of the set of extrapolated tracks.

12 . The system of claim 8 , wherein the input data comprises camera data obtained from at least one camera of the AV and radar data obtained from at least one radar of the AV.

13 . The system of claim 8 , wherein, to obtain the input data, the processing device is further configured to generate a dataset with lane labels using auto-labeling.

14 . The system of claim 8 , wherein, to obtain the set of polylines, the processing device is further configured to:

extract the set of polylines from the at least one heatmap; or

generate, using a generative model, the set of polylines using the at least one heatmap.

15 . A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by a processing device, cause the processing device to perform operations comprising:

obtaining input data derived from a set of sensors of an autonomous vehicle (AV);

generating at least one heatmap in a bird's eye view (BEV) representation by applying a set of lane detection classifier heads to a fused BEV feature generated from the input data;

obtaining a set of polylines in the BEV representation using the at least one heatmap, wherein each polyline of the set of polylines corresponds to a respective track of a first set of tracks for a first frame; and

generating a second set of tracks for a second frame after the first frame by using a statistical filter based on a set of extrapolated tracks for the second frame and a set of track measurements for the second frame, wherein each track measurement of the set of track measurements corresponds to a respective updated polyline obtained for the second frame.

16 . The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

extrapolating each track of the first set of tracks to generate the set of extrapolated tracks for the second frame; and

obtaining the set of track measurements for the second frame.

17 . The non-transitory computer-readable storage medium of claim 15 , wherein generating the second set of tracks further comprises:

associating a track measurement of the set of track measurements to an extrapolated track of the set of extrapolated tracks, wherein the track measurement comprises a set of measured points;

projecting a set of points of the extrapolated track to the track measurement to obtain a set of projected points of the track measurement; and

generating an updated set of points for a track of the second set of tracks based on the set of projected points and the set of measured points.

18 . The non-transitory computer-readable storage medium of claim 15 , wherein generating the second set of tracks further comprises creating a new track for a track measurement that is not associated with an extrapolated track of the set of extrapolated tracks.

19 . The non-transitory computer-readable storage medium of claim 15 , wherein the input data comprises camera data obtained from at least one camera of the AV and radar data obtained from at least one radar of the AV, and wherein obtaining the input data further comprises generating a dataset with lane labels using auto-labeling.

20 . The non-transitory computer-readable storage medium of claim 15 , wherein obtaining the set of polylines further comprises one of:

extracting the set of polylines from the at least one heatmap; or

generating, using a generative model, the set of polylines using the at least one heatmap.