Object auto-labeling
Techniques for determining labels for potentially non-impeding objects in an environment are disclosed. Unlabeled lidar segments may be evaluated to determine whether they are located in a drivable road in an environment and to determine a lidar intensity value for the segments. Based on the intensity and whether the segment is in a drivable region, the system determines a corresponding range of lidar values associated with a label or no label. The system assigns the label (or no label) associated with the corresponding range. The label and associated segment data may then be used to classify an object associated with the segment, control a vehicle, and train a machine-learned model.
1 . A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
receiving lidar data associated with an environment, the lidar data comprising one or more labeled lidar segments and one or more unlabeled lidar segments;
determining one or more first lidar segments associated with a drivable region of the environment from the one or more unlabeled lidar segments;
determining one or more second lidar segments associated with a non-drivable region of the environment from the one or more unlabeled lidar segments;
determining an intensity value for a first lidar segment of the one or more first lidar segments;
determining an intensity value for a second lidar segment of the one or more second lidar segments;
determining a first lidar intensity value range associated with the drivable region of the environment and corresponding to the intensity value for the first lidar segment;
determining a second lidar intensity value range associated with the non-drivable region of the environment and corresponding to the intensity value for the second lidar segment;
assigning a first label to the first lidar segment based at least in part on the first lidar intensity value range;
assigning a second label to the second lidar segment based at least in part on the second lidar intensity value range;
assigning no label to a third lidar segment of at least one of the one or more first lidar segments or the one or more second lidar segments based at least in part on determining that a third intensity value associated with the third lidar segment is within a third lidar intensity value range associated with one of the drivable region of the environment or the non-drivable region of the environment; and
controlling a vehicle based at least in part on the first label and the second label.
2 . The system of claim 1 , wherein:
the third lidar intensity value range comprises values greater than values of the first lidar intensity value range and less than values of a fourth lidar intensity value range associated with the drivable region of the environment.
3 . The system of claim 1 , wherein determining the first lidar intensity value range corresponding to the intensity value for the first lidar segment comprises:
determining a plurality of lidar intensity value ranges associated with the drivable region of the environment, the plurality of lidar intensity value ranges comprising the first lidar intensity value range; and
determining that the intensity value for the first lidar segment is represented among values of the first lidar intensity value range.
4 . The system of claim 1 , wherein determining the second lidar intensity value range corresponding to the intensity value for the second lidar segment comprises:
determining a plurality of lidar intensity value ranges associated with the non-drivable region of the environment, the plurality of lidar intensity value ranges comprising the first lidar intensity value range and the third lidar intensity value range, wherein values of the second lidar intensity value range are greater than values of the third lidar intensity value range; and
determining that the intensity value for the second lidar segment is represented among the values of the second lidar intensity value range.
5 . The system of claim 1 , wherein the environment is a first environment and the operations further comprise transmitting a machine-learned (ML) model trained to detect a non-impeding object based at least in part on the first label, the first lidar segment, the second label, and the second lidar segment to a second vehicle configured to traverse a second environment based at least in part on output received from the ML model.
6 . A method comprising:
determining lidar data associated with an environment, the lidar data comprising a first unlabeled lidar segment and a second unlabeled lidar segment;
determining that the first unlabeled lidar segment is associated with a drivable region of the environment;
determining a first intensity value associated with the first unlabeled lidar segment;
determining a plurality of intensity value ranges based at least in part on determining that the first unlabeled lidar segment is associated with the drivable region of the environment;
assigning a label to the first unlabeled lidar segment based at least in part on determining that the first intensity value is within a first intensity value range of the plurality of intensity value ranges;
assigning no label to the second unlabeled lidar segment based at least in part on determining that a second intensity value associated with the second unlabeled lidar segment is within a second intensity value range of the plurality of intensity value ranges; and
controlling a vehicle based at least in part on the label.
7 . The method of claim 6 , wherein:
the first intensity value range is a highest intensity value range among the plurality of intensity value ranges; and
the label indicates that the first unlabeled lidar segment is associated with an impeding object.
8 . The method of claim 6 , wherein:
the first intensity value range is a lowest intensity value range among the plurality of intensity value ranges; and
the label indicates that the first unlabeled lidar segment is associated with a non-impeding object.
9 . The method of claim 6 , wherein determining the lidar data comprises:
receiving sensor data from a plurality of sensors; and
determining a plurality of two-dimensional lidar segments comprising the first unlabeled lidar segment based at least in part on the sensor data.
10 . The method of claim 6 , wherein the plurality of intensity value ranges comprises:
a lowest range associated with non-impeding objects;
a moderate range associated with no label; and
a highest range associated with impeding objects.
11 . The method of claim 10 , wherein values of the moderate range are greater than values of the lowest range and less than values of a highest range.
12 . The method of claim 6 , further comprising:
determining one or more lidar points represented in the lidar data that are not associated with one or more lidar segments represented in the lidar data; and
filtering the one or more lidar points from the lidar data.
13 . The method of claim 6 , further comprising training a machine-learned (ML) model to detect a non-impeding object based at least in part on labeled lidar data comprising the label and data associated with the first unlabeled lidar segment.
14 . The method of claim 13 , further comprising transmitting the ML model to a second vehicle which is configured to be controlled based at least in part on output received from the ML model.
15 . One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving lidar data associated with an environment, the lidar data comprising a first unlabeled lidar segment and a second unlabeled lidar segment;
determining that the first unlabeled lidar segment is associated with a non-drivable region of the environment;
determining a first intensity value associated with the first unlabeled lidar segment;
determining a plurality of intensity ranges based at least in part on determining that the first unlabeled lidar segment is associated with the non-drivable region of the environment;
assigning a label to the first unlabeled lidar segment based at least in part determining that the first intensity value is associated with a first intensity range of the plurality of intensity ranges;
assigning no label to the second unlabeled lidar segment based at least in part on determining that a second intensity value associated with the second unlabeled lidar segment is within a second intensity range of the plurality of intensity ranges; and
controlling a vehicle based at least in part on the label.
16 . The one or more non-transitory computer-readable media of claim 15 , wherein:
the second unlabeled lidar segment is associated with a drivable region of the environment.
17 . The one or more non-transitory computer-readable media of claim 15 , wherein:
the first intensity range is a highest intensity range among the plurality of intensity ranges; and
the second intensity range is a lowest intensity range among the plurality of intensity ranges.
18 . The one or more non-transitory computer-readable media of claim 15 , wherein the first intensity range comprises one or more of:
a range of intensity values; or
a range of intensity percentages.
19 . The one or more non-transitory computer-readable media of claim 18 , wherein determining the plurality of intensity ranges comprises determining the plurality of intensity ranges from among a plurality of ranges comprising a non-drivable region plurality of intensity ranges and a drivable region plurality of intensity ranges.
20 . The one or more non-transitory computer-readable media of claim 15 , wherein the operations further comprise training a machine-learned (ML) model to detect a non-impeding object based at least in part on the first unlabeled lidar segment and determining whether to assign the label to the first unlabeled lidar segment.