Automatic generation of ground truth data for training or retraining machine learning models
In various examples, object detections of a machine learning model are leveraged to automatically generate new ground truth data for images captured at different perspectives. The machine learning model may generate a prediction of a detected object at the different perspective, and an object tracking algorithm may be used to track the object through other images in a sequence of images where the machine learning model may not have detected the object. New ground truth data may be generated as a result of the object tracking algorithms outputs, and the new ground truth data may be used to retrain or update the machine learning model, train a different machine learning model, or increase the robustness of a ground truth data set that may be used for training machine learning models from various perspectives.
1. A method comprising:
applying a sequence of images to one or more machine learning models (MLMs) trained to generate predictions corresponding to object detection for at least one object class;
detecting, using one or more of the predictions, an object corresponding to the at least one object class in at least one image of the sequence of images;
tracking the object from the at least one image through one or more other images from the sequence of images to determine a presence of the object in the one or more other images; and
automatically associating at least one object label associated with the object with the one or more other images based at least on the determining of the presence of the object in the one or more other images, wherein one or more predictions corresponding to object detection generated using the one or more MLMs are used to perform one or more operations for at least one or more of:
an augmented reality application, a virtual reality application, a robotics application, a security and surveillance application, a character recognition application, a medical imaging application, a simulation application, a synthetic data generation application, or an autonomous machine application.
2. The method of claim 1 , further comprising selecting at least an image of the one or more other images for the tracking based at least on the image comprising a false negative detection of the object by the one or more MLMs.
3. The method of claim 1 , wherein the tracking the object from the at least one image through the one or more other images is in a reverse ordering of the one or more other images in the sequence of images.
4. The method of claim 1 , comprising re-arranging an ordering of the one or more other images in the sequence of images, wherein the tracking of the object is through the re-arranged ordering of the one or more other images.
5. The method of claim 1 , further comprising updating one or more parameters of at least one of the one or more MLMs or at least one other MLM using the at least one object label as ground truth data.
6. The method of claim 1 , wherein the sequence of images is generated using at least one first image sensor having at least one first perspective, and the one or more MLMs were trained on images generated using at least one second image sensor having at least one second perspective different from the at least one first perspective.
7. The method of claim 1 , wherein the tracking includes using one or more bounding labels for the object in the at least one image to determine the presence of the object in the one or more other images.
8. The method of claim 1 , wherein the tracking includes using one or more locations of the object in the at least one image to determine a location of the object in at least an image of the one or more other images.
9. The method of claim 1 , further comprising deploying at least one other MLM trained using the at least one object label, wherein a plurality of object detections made using the at least one other MLM are used to perform the one or more operations for the at least one of:
the augmented reality application, the virtual reality application, the robotics application, the security and surveillance application, the character recognition application, the medical imaging application, the simulation application, the synthetic data generation application, or the autonomous machine application.
10. A system comprising:
one or more processing units to execute operations comprising:
applying image data to one or more machine learning models (MLMs);
detecting, using the one or more MLMs, an object in one or more first images from a set of images represented by the image data;
tracking the object from the one or more first images through one or more second images from the set of images to detect the object in the one or more second images; and
generating ground truth data associating the object with the one or more second images based at least on the detecting of the object in the one or more second images, wherein one or more predictions corresponding to object detection generated using the one or more MLMs are used to perform one or more operations for at least one or more of:
an augmented reality application, a virtual reality application, a robotics application, a security and surveillance application, a character recognition application, a medical imaging application, a simulation application, a synthetic data generation application, or an autonomous machine application.
11. The system of claim 10 , wherein the operations further comprise selecting at least one image of the one or more second images for the tracking of the object based at least on each image of the at least one image comprising a false negative detection of the object by the one or more MLMs.
12. The system of claim 10 , wherein the tracking includes using one or more bounding shapes for the object in the one or more first images to determine a presence of the object in the one or more second images.
13. The system of claim 10 , wherein the tracking of the object is through a plurality of the one or more second images in a re-arranged ordering relative to an order in which the set of images is applied to the one or more MLMs.
14. The system of claim 10 , wherein the set of images is generated using at least one first image sensor having at least one first perspective, and the one or more MLMs were trained on images generated using at least one second image sensor having at least one second perspective different from the at least one first perspective.
15. The system of claim 10 , wherein the operations further comprise deploying at least one other MLM trained using the ground truth data, wherein a plurality of object detections made using the at least one other MLM are used to perform the one or more operations for the at least one of:
the augmented reality application, the virtual reality application, the robotics application, the security and surveillance application, the character recognition application, the medical imaging application, the simulation application, the synthetic data generation application, or the autonomous machine application.
16. A processor comprising:
one or more circuits to track an object through one or more first images from a set of images applied to one or more machine learning models (MLMs) using one or more second images from the set of images based at least on the one or more MLMs indicating a presence of the object in the one or more second images, and to generate one or more labels for the one or more first images based at least on the tracking of the object through the one or more second images, wherein one or more predictions corresponding to object detection generated using the one or more MLMs are used to perform one or more operations for at least one or more of:
an augmented reality application, a virtual reality application, a robotics application, a security and surveillance application, a character recognition application, a medical imaging application, a simulation application, a synthetic data generation application, or an autonomous machine application.
17. The processor of claim 16 , wherein the one or more circuits are further to select at least one image of the one or more first images for the detecting of the object based at least on each image of the at least one image comprising a false negative detection of the object by the one or more MLMs.
18. The processor of claim 16 , wherein the tracking includes using one or more bounding shapes for the object in the one or more second images to determine a presence of the object in the one or more first images.
19. The processor of claim 16 , wherein the tracking of the object is through a plurality of the one or more first images in a re-arranged ordering relative to an order the set of images is applied to the one or more MLMs.
20. The processor of claim 16 , wherein the one or more circuits are further to deploy at least one other MLM trained using the at least one object label, wherein a plurality of object detections made using the at least one other MLM are used to perform the one or more operations for the at least one of:
the augmented reality application, the virtual reality application, the robotics application, the security and surveillance application, the character recognition application, the medical imaging application, the simulation application, the synthetic data generation application, or the autonomous machine application.