System and method for actively selecting and labeling images for semantic segmentation
A system and method for actively selecting and labeling images for semantic segmentation are disclosed. A particular embodiment includes: receiving image data from an image generating device; performing semantic segmentation or other object detection on the received image data to identify and label objects in the image data and produce semantic label image data; determining the quality of the semantic label image data based on prediction probabilities associated with regions or portions of the image; and identifying a region or portion of the image for manual labeling if an associated prediction probability is below a pre-determined threshold.
1. A system comprising:
a processor configured to:
receive image data corresponding to an image from an image generating device;
perform object detection on the received image data to produce semantic label image data by identifying and labeling objects in a plurality of regions of the image;
determine prediction probabilities associated with the plurality of regions of the image, wherein the prediction probabilities indicate likelihood that the objects in the plurality of regions are identified relative to training data;
identify a region of the image for manual labeling in response to determining that a prediction probability associated with the region of the image is below a pre-determined threshold; and
generate a map that shows portions of the image data having prediction probabilities below the pre-determined threshold, and wherein the portions include the identified region.
2. The system of claim 1 wherein the image generating device is one or more cameras.
3. The system of claim 1 wherein the image data corresponds to at least one frame from a video stream generated by one or more cameras.
4. The system of claim 1 wherein the processor is further configured to retrain the object detection process based on previously generated semantic label image data.
5. The system of claim 1 wherein the processor is further configured to refine and label the objects of the image by being configured to combine manually-generated label image data with the semantic label image data.
6. The system of claim 1 wherein the prediction probabilities are determined for each pixel of the received image data.
7. The system of claim 1 wherein the portions of the map are generated using a black color.
8. A method comprising:
receiving image data corresponding to an image from an image generating device;
performing object detection on the received image data to produce semantic label image data by identifying and labeling objects in a plurality of regions of the image;
determining prediction probabilities associated with the plurality of regions of the image, wherein the prediction probabilities indicate likelihood that the objects in the plurality of regions are identified relative to training data;
identifying a region of the image for manual labeling in response to determining that a prediction probability associated with the region of the image is below a pre-determined threshold; and
generating a map that shows portions of the image data having prediction probabilities below the pre-determined threshold, and wherein the portions include the identified region.
9. The method of claim 8 wherein the image generating device is one or more cameras.
10. The method of claim 8 wherein the image data corresponds to at least one frame from a video stream generated by one or more cameras.
11. The method of claim 8 including retraining the object detection process based on previously generated semantic label image data.
12. The method of claim 8 further comprising:
refining and labeling the objects of the image by combining manually-generated label image data with the semantic label image data.
13. The method of claim 8 wherein the prediction probabilities are determined for each pixel of the received image data.
14. The method of claim 8 wherein the portions of the map are generated using a black color.
15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive image data corresponding to an image from an image generating device;
perform object detection on the received image data to produce semantic label image data by identifying and labeling objects in a plurality of regions of the image;
determine prediction probabilities associated with the plurality of regions of the image, wherein the prediction probabilities indicate likelihood that the objects in the plurality of regions are identified relative to training data;
identify a region of the image for manual labeling in response to determining that a prediction probability associated with the region of the image is below a pre-determined threshold; and
generate a map that shows portions of the image data having prediction probabilities below the pre-determined threshold, and wherein the portions include the identified region.
16. The non-transitory machine-useable storage medium of claim 15 wherein the image generating device is one or more cameras.
17. The non-transitory machine-useable storage medium of claim 15 wherein the image data corresponds to at least one frame from a video stream generated by one or more cameras.
18. The non-transitory machine-useable storage medium of claim 15 wherein the machine is further configured to retrain the object detection process based on previously generated semantic label image data.
19. The non-transitory machine usable storage medium of claim 15 , wherein the machine is further configured to refine and label the objects of the image by being configured to combine manually-generated label image data with the semantic label image data.
20. The non-transitory machine usable storage medium of claim 15 wherein the portions of the map are generated using a black color.