System and method for multitask processing for autonomous vehicle computation and control
A system and method for multitask processing for autonomous vehicle computation and control includes: receiving training image data from a training image data collection system; performing a training phase to train a plurality of tasks associated with features of the training image data, the training phase including extracting common features from the training image data, causing the plurality of tasks to generate task-specific predictions based on the training image data, determining a bias between the task-specific prediction for each task and corresponding task-specific ground truth data, and adjusting parameters of each of the plurality of tasks to cause the bias to meet a pre-defined confidence level; receiving image data from an image data collection system associated with an autonomous vehicle; and performing an operational phase including extracting common features from the image data, causing the plurality of trained tasks to concurrently generate task-specific predictions based on the image data.
1. A system comprising:
a data processor; and
a memory for storing an autonomous vehicle computation and control system, executable by the data processor, the autonomous vehicle computation and control system being configured to:
receive training image data from a training image data collection system, the training image data collection system comprising sensors installed in a moving test vehicle navigated through real-world traffic scenarios, the training image data comprising data collected from real-world traffic scenarios;
obtain ground truth data corresponding to the training image data, the ground truth data corresponding to the data collected from real-world traffic scenarios;
perform a training phase to train a plurality of tasks each associated with different features of the training image data, at least one task of the plurality of tasks corresponding to a specific feature of the training image data, the plurality of tasks configured to execute concurrently, the training phase comprising extracting features from the training image data, associating similar extracted features with a corresponding task of the plurality of tasks, associating different extracted features with different other tasks of the plurality of tasks, causing the plurality of tasks to generate task-specific predictions of feature characteristics based on the training image data, determining a bias between the task-specific prediction for each task and corresponding task-specific ground truth data, and adjusting parameters of each of the plurality of tasks to cause the bias to meet a pre-defined confidence level;
receive image data from an image data collection system associated with an autonomous vehicle; and
perform an operational phase comprising extracting features from the image data, causing the plurality of trained tasks to execute concurrently to generate task-specific predictions of feature characteristics based on the image data, and output the task-specific predictions to an autonomous vehicle subsystem of the autonomous vehicle.
2. The system of claim 1 being further configured to use machine learnable components.
3. The system of claim 1 wherein the training image data collection system comprises a video camera and a light detection and ranging (LIDAR) device.
4. The system of claim 1 wherein the system being configured to obtain ground truth data from a manual annotation data collection system, which comprises gathering information from human labelers.
5. The system of claim 1 wherein the system being further configured to receive a plurality of training images gathered from a plurality of perception devices by the training image data collection system.
6. The system of claim 1 wherein the system being further configured to receive a plurality of images gathered from a plurality of perception devices at various traffic locations by the image data collection system.
7. The system of claim 1 wherein the system being further configured to provide highly accurate image feature predictions for configuring a control system in an autonomous vehicle.
8. A method comprising:
receiving training image data from a training image data collection system, the training image data collection system comprising sensors installed in a moving test vehicle navigated through real-world traffic scenarios, the training image data comprising data collected from real-world traffic scenarios;
obtaining ground truth data corresponding to the training image data, the ground truth data corresponding to the data collected from real-world traffic scenarios;
performing a training phase to train a plurality of tasks each associated with different features of the training image data, at least one task of the plurality of tasks corresponding to a specific feature of the training image data, the plurality of tasks configured to execute concurrently, the training phase comprising extracting features from the training image data, associating similar extracted features with a corresponding task of the plurality of tasks, associating different extracted features with different other tasks of the plurality of tasks, causing the plurality of tasks to generate task-specific predictions of feature characteristics based on the training image data, determining a bias between the task-specific prediction for each task and corresponding task-specific ground truth data, and adjusting parameters of each of the plurality of tasks to cause the bias to meet a pre-defined confidence level;
receiving image data from an image data collection system associated with an autonomous vehicle; and
performing an operational phase comprising extracting features from the image data, causing the plurality of trained tasks to execute concurrently to generate task-specific predictions of feature characteristics based on the image data, and output the task-specific predictions to an autonomous vehicle subsystem of the autonomous vehicle.
9. The method of claim 8 further comprises further comprising extracting features to provide a pixel-level object label for each feature.
10. The method of claim 8 wherein the determined bias comprises a computed loss value.
11. The method of claim 8 further comprising obtaining ground truth data from an automated process.
12. The method of claim 8 further comprising associating a weight to each of the plurality of tasks.
13. The method of claim 8 further comprising continuing to train a task of the plurality of tasks while the bias fails to meet a pre-defined confidence level.
14. The method of claim 8 wherein the image data collection system comprises perception devices, wherein the perception devices comprise cameras and sensors.
15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive training image data from a training image data collection system, the training image data collection system comprising sensors installed in a moving test vehicle navigated through real-world traffic scenarios, the training image data comprising data collected from real-world traffic scenarios;
obtain ground truth data corresponding to the training image data, the ground truth data corresponding to the data collected from real-world traffic scenarios;
perform a training phase to train a plurality of tasks each associated with different features of the training image data, at least one task of the plurality of tasks corresponding to a specific feature of the training image data, the plurality of tasks configured to execute concurrently, the training phase comprising extracting features from the training image data, associating similar extracted features with a corresponding task of the plurality of tasks, associating different extracted features with different other tasks of the plurality of tasks, causing the plurality of tasks to generate task-specific predictions of feature characteristics based on the training image data, determining a bias between the task-specific prediction for each task and corresponding task-specific ground truth data, and adjusting parameters of each of the plurality of tasks to cause the bias to meet a pre-defined confidence level;
receive image data from an image data collection system associated with an autonomous vehicle; and
perform an operational phase comprising extracting features from the image data, causing the plurality of trained tasks to execute concurrently to generate task-specific predictions of feature characteristics based on the image data, and output the task-specific predictions to an autonomous vehicle subsystem of the autonomous vehicle.
16. The non-transitory machine-useable storage medium of claim 15 wherein the instructions further comprise a neural network.
17. The non-transitory machine-useable storage medium of claim 15 wherein the instructions are further configured to obtain ground truth data from a manual annotation data collection system, which comprises gathering information from human labelers and automated processes.
18. The non-transitory machine-useable storage medium of claim 15 wherein the instructions are further configured to receive a plurality of training images gathered from a plurality of perception devices by the training image data collection system, the plurality of perception devices comprising cameras and sensors.
19. The non-transitory machine-useable storage medium of claim 15 wherein the instructions are further configured to extract features using semantic segmentation.
20. The non-transitory machine-useable storage medium of claim 15 wherein the plurality of tasks are associated with shared layers among different pixel-level image segmentation.