Automatic data labelling for autonomous driving vehicles
In one embodiment, sensor data are collected from one or more sensors mounted on an autonomous driving vehicle (ADV) while the ADV is moving within a region of interest (ROI) that includes a number of obstacles. The collected sensor data are operated on to obtain obstacle data associated with the obstacles, location data, and a number of timestamps that correspond to the obstacle data and the location data. For each of the timestamps, positions of the obstacles are mapped to some of the obstacle data that correspond to the timestamp based on the location data, thereby generating mapped information of the obstacles. The mapped information is automatically labelled to generate labelled data, where the labelled data is utilized to subsequently train a machine learning algorithm to recognize obstacles during autonomous driving of an ADV.
1. A computer-implemented method for automatic generation of labelled data, comprising:
pre-marking a region of interest (ROI) and a plurality of obstacles within the ROI from an initial position of an autonomous driving vehicle (ADV) to generate pre-marked data;
collecting sensor data from one or more sensors mounted on the ADV while the ADV is moving within the ROI;
processing the collected sensor data to obtain obstacle data associated with the obstacles, location data, and a plurality of timestamps that correspond to the obstacle data and the location data;
for each of the timestamps, mapping positions of the obstacles to some of the obstacle data that correspond to the timestamp based on the location data and the pre-marked data, thereby generating mapped information of the obstacles; and
automatically labelling the mapped information to generate labelled data, wherein the labelled data is utilized to subsequently train a machine learning algorithm to recognize obstacles during autonomous driving of the ADV.
2. The method of claim 1 , wherein pre-marking the ROI and the plurality of obstacles within the ROI comprises prior to driving the ADV within the ROI,
measuring and recording a boundary of the ROI based on current location data of the ADV, and
measuring and recording an initial position and a boundary of each obstacle within the ROI based on current radar data or current light detection and range (LIDAR) data of the ADV,
wherein the pre-marked data comprises the recorded boundary of the ROI, and the recorded initial position and boundary of each obstacle within the ROI.
3. The method of claim 1 , further comprising prior to mapping the positions of the obstacles to some of the obstacle data that correspond to the timestamp, aligning the plurality of timestamps and the positions of the obstacles.
4. The method of claim 1 , wherein the obstacle data comprises light detection and range (LIDAR) data captured at different points in time that are included in the timestamps.
5. The method of claim 4 , wherein the location data comprises the positions of the obstacles captured at the different points in time.
6. The method of claim 5 , wherein the location data is obtained from an inertial measurement unit (IMU) or a global positioning system (GPS).
7. The method of claim 1 , wherein the pre-marked data comprises boundary information of the ROI, and initial position and boundary information of the plurality of obstacles within the ROI.
8. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
pre-marking a region of interest (ROI) and a plurality of obstacles within the ROI from an initial position of an autonomous driving vehicle (ADV) to generate pre-marked data;
collecting sensor data from one or more sensors mounted on the ADV while the ADV is moving within the ROI;
processing the collected sensor data to obtain obstacle data associated with the obstacles, location data, and a plurality of timestamps that correspond to the obstacle data and the location data;
for each of the timestamps, mapping positions of the obstacles to some of the obstacle data that correspond to the timestamp based on the location data and the pre-marked data, thereby generating mapped information of the obstacles; and
automatically labelling the mapped information to generate labelled data, wherein the labelled data is utilized to subsequently train a machine learning algorithm to recognize obstacles during autonomous driving of the ADV.
9. The non-transitory machine-readable medium of claim 8 , wherein pre-marking the ROI and the plurality of obstacles within the ROI comprises prior to driving the ADV within the ROI,
measuring and recording a boundary of the ROI based on current location data of the ADV, and
measuring and recording an initial position and a boundary of each obstacle within the ROI based on current radar data or current light detection and range (LIDAR) data of the ADV,
wherein the pre-marked data comprises the recorded boundary of the ROI, and the recorded initial position and boundary of each obstacle within the ROI.
10. The non-transitory machine-readable medium of claim 8 , wherein the operations further comprise: prior to mapping the positions of the obstacles to some of the obstacle data that correspond to the timestamp, aligning the plurality of timestamps and the positions of the obstacles.
11. The non-transitory machine-readable medium of claim 8 , wherein the obstacle data comprises light detection and range (LIDAR) data captured at different points in time that are included in the timestamps.
12. The non-transitory machine-readable medium of claim 11 , wherein the location data comprises the positions of the obstacles captured at the different points in time.
13. The non-transitory machine-readable medium of claim 12 , wherein the location data is obtained from an inertial measurement unit (IMU) or a global positioning system (GPS).
14. The non-transitory machine-readable medium of claim 8 , wherein the pre-marked data comprises boundary information of the ROI, and initial position and boundary information of the plurality of obstacles within the ROI.
15. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including
pre-marking a region of interest (ROI) and a plurality of obstacles within the ROI from an initial position of an autonomous driving vehicle (ADV) to generate pre-marked data:
collecting sensor data from one or more sensors mounted on the ADV while the ADV is moving within the ROI;
processing the collected sensor data to obtain obstacle data associated with the obstacles, location data, and a plurality of timestamps that correspond to the obstacle data and the location data;
for each of the timestamps, mapping positions of the obstacles to some of the obstacle data that correspond to the timestamp based on the location data and the pre-marked data, thereby generating mapped information of the obstacles; and
automatically labelling the mapped information to generate labelled data, wherein the labelled data is utilized to subsequently train a machine learning algorithm to recognize obstacles during autonomous driving of the ADV.
16. The data processing system of claim 15 , wherein pre-marking the ROI and the plurality of obstacles within the ROI comprises prior to driving the ADV within the ROI,
measuring and recording a boundary of the ROI based on current location data of the ADV, and
measuring and recording an initial position and a boundary of each obstacle within the ROI based on current radar data or current light detection and range (LIDAR) data of the ADV,
wherein the pre-marked data comprises the recorded boundary of the ROI, and the recorded initial position and boundary of each obstacle within the ROI.
17. The data processing system of claim 15 , wherein the operations further include: prior to mapping the positions of the obstacles to some of the obstacle data that correspond to the timestamp, aligning the plurality of timestamps and the positions of the obstacles.
18. The data processing system of claim 15 , wherein the obstacle data comprises light detection and range (LIDAR) data captured at different points in time that are included in the timestamps.
19. The data processing system of claim 18 , wherein the location data comprises the positions of the obstacles captured at the different points in time.
20. The data processing system of claim 19 , wherein the location data is obtained from an inertial measurement unit (IMU) or a global positioning system (GPS).
21. The data processing system of claim 15 , wherein the pre-marked data comprises boundary information of the ROI, and initial position and boundary information of the plurality of obstacles within the ROI.