System and method for instance-level lane detection for autonomous vehicle control
A system and method for instance-level roadway feature detection for autonomous vehicle control are disclosed. A particular embodiment includes: receiving image data from an image data collection system associated with an autonomous vehicle; extracting roadway features from the image data, causing a plurality of trained tasks to generate instance-level roadway feature detection results based on the image data, the plurality of trained tasks having been individually trained with different features of training image data received from a training image data collection system and corresponding ground truth data, the training image data and the ground truth data comprising data collected from real-world traffic scenarios; causing the plurality of trained tasks to generate task-specific predictions of feature characteristics based on the image data and to generate corresponding instance-level roadway feature detection results; and providing the instance-level roadway feature detection results to an autonomous vehicle subsystem to control operation of the autonomous vehicle.
1. A system comprising:
a data processor; and
a memory for storing an autonomous vehicle instance-level roadway feature detection system, executable by the data processor, the autonomous vehicle instance-level roadway feature detection system being configured to:
receive image data from an image data collection system associated with an autonomous vehicle;
extract roadway features from the image data, causing a plurality of trained tasks to generate instance-level roadway feature detection results based on the image data, the plurality of trained tasks having been individually trained with different features of training image data received from a training image data collection system and corresponding ground truth data, the training image data and the ground truth data comprising data collected from real-world traffic scenarios;
cause the plurality of trained tasks to generate task-specific predictions of feature characteristics based on the image data and to generate corresponding instance-level roadway feature detection results, the instance-level roadway feature detection results including a feature map, which identifies and differentiates each of a plurality of different roadway feature instances present in the image data;
provide the instance-level roadway feature detection results to an autonomous vehicle subsystem of the autonomous vehicle; and
control operation of the autonomous vehicle based on the instance-level roadway feature detection results.
2. The system of claim 1 wherein the plurality of different roadway feature instances corresponds to different instances of a same roadway feature type.
3. The system of claim 1 being further configured to generate a roadway feature object map and a roadway feature instance map based on the extracted roadway features.
4. The system of claim 1 wherein the image data collection system comprises sensors installed in the autonomous vehicle.
5. The system of claim 1 being further configured to receive a plurality of images gathered from a plurality of perception devices by the image data collection system.
6. The system of claim 1 being further configured to receive sensor data from a plurality of perception devices.
7. The system of claim 1 wherein at least one trained task of the plurality of trained tasks corresponding to a specific feature from the image data.
8. A method for autonomous vehicle instance-level roadway feature detection, the method comprising:
receiving image data from an image data collection system associated with an autonomous vehicle;
extracting roadway features from the image data, causing a plurality of trained tasks to generate instance-level roadway feature detection results based on the image data, the plurality of trained tasks having been individually trained with different features of training image data received from a training image data collection system and corresponding ground truth data, the training image data and the ground truth data comprising data collected from real-world traffic scenarios;
causing the plurality of trained tasks to generate task-specific predictions of feature characteristics based on the image data and to generate corresponding instance-level roadway feature detection results, the instance-level roadway feature detection results including a feature map, which identifies and differentiates each of a plurality of different roadway feature instances present in the image data;
providing the instance-level roadway feature detection results to an autonomous vehicle subsystem of the autonomous vehicle; and
controlling operation of the autonomous vehicle based on the instance-level roadway feature detection results.
9. The method of claim 8 further comprising extracting features to provide a pixel-level object label for each roadway feature.
10. The method of claim 8 further comprising generating a roadway feature object map and a roadway feature instance map based on the extracted roadway features and determining a computed loss value with a bias.
11. The method of claim 8 including associating similar extracted features with a corresponding trained task of the plurality of trained tasks, and associating different extracted features with different other trained tasks of the plurality of trained tasks.
12. The method of claim 8 further comprising receiving a plurality of images gathered from a plurality of perception devices by the image data collection system and associating a weight to each of the plurality of trained tasks.
13. The method of claim 8 further comprising receiving sensor data from a plurality of perception devices, the plurality of perception devices comprising cameras and sensors.
14. The method of claim 8 further comprising configuring a control system in an autonomous vehicle based on results produced by the plurality of trained tasks.
15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive image data from an image data collection system associated with an autonomous vehicle;
extract roadway features from the image data, causing a plurality of trained tasks to generate instance-level roadway feature detection results based on the image data, the plurality of trained tasks having been individually trained with different features of training image data received from a training image data collection system and corresponding ground truth data, the training image data and the ground truth data comprising data collected from real-world traffic scenarios;
cause the plurality of trained tasks to generate task-specific predictions of feature characteristics based on the image data and to generate corresponding instance-level roadway feature detection results, the instance-level roadway feature detection results including a feature map, which identifies and differentiates each of a plurality of different roadway feature instances present in the image data;
provide the instance-level roadway feature detection results to an autonomous vehicle subsystem of the autonomous vehicle; and
control operation of the autonomous vehicle based on the instance-level roadway feature detection results.
16. The non-transitory machine-useable storage medium of claim 15 being further configured to detect different instances of a same roadway feature type.
17. The non-transitory machine-useable storage medium of claim 15 being further configured to generate a roadway feature object map and a roadway feature instance map based on the extracted roadway features using a neural network.
18. The non-transitory machine-useable storage medium of claim 15 being further configured to receive a plurality of training images gathered from a plurality of perception devices by a training image data collection system and processed by human labelers and automated processes.
19. The non-transitory machine-useable storage medium of claim 15 being further configured to:
receive training image data from a training image data collection system, the training image data collection system comprising sensors installed in a moving test vehicle navigated through real-world traffic scenarios, the training image data comprising data collected from real-world traffic scenarios; and
perform a training phase to train the plurality of tasks each associated with different features of the training image data, at least one task of the plurality of tasks corresponding to a specific feature of the training image data, the training phase comprising extracting roadway features from the training image data, associating similar extracted features with a corresponding task of the plurality of tasks, associating different extracted features with different other tasks of the plurality of tasks, causing the plurality of tasks to generate task-specific predictions of feature characteristics based on the training image data, determining a bias between the task-specific prediction for each task and corresponding task-specific ground truth data, and adjusting parameters of each of the plurality of tasks to cause the bias to meet a pre-defined confidence level.
20. The non-transitory machine-useable storage medium of claim 15 wherein the plurality of tasks are associated with shared layers among different pixel-level image segmentation.