Aggregated image annotation
Image annotation includes: accessing an image and a plurality of annotation data sets for the image, wherein the plurality of annotation data sets are made by a plurality of contributors, and the image has a plurality of original image channels; aggregating the plurality of annotation data sets to obtain an aggregated annotation data set for the image; and outputting the aggregated annotation data set. Aggregating the plurality of annotation data sets to obtain an aggregated annotation data set for the image includes: generating an additional image channel based at least in part on weight averages of confidence measures of the plurality of contributors; and applying an object detection model to at least a part of the plurality of original image channels and at least a part of the additional image channel to generate the aggregated annotation data set.
1. A method of image annotation, comprising:
accessing an image and a plurality of annotation data sets for the image, wherein the plurality of annotation data sets are made by a plurality of contributors, and the image has a plurality of original image channels;
aggregating the plurality of annotation data sets to obtain an aggregated annotation data set for the image, including:
generating an additional image channel based at least in part on weight averages of confidence measures of the plurality of contributors, wherein a confidence measure of a contributor indicates an accuracy of the contributor in making annotations or how confident the contributor is at making an annotation; and
applying an object detection model to at least a part of the plurality of original image channels and at least a part of the additional image channel to generate the aggregated annotation data set; and
outputting the aggregated annotation data set.
2. The method of claim 1 , wherein the generating of the additional image channel includes determining, for a pixel in the image, a sum of associated values with respect to one or more bounding boxes.
3. The method of claim 2 , wherein the associated values with respect to the one or more bounding boxes are associated with the confidence measures of the plurality of contributors.
4. The method of claim 1 , wherein the generating of the additional image channel includes, for a pixel in the image, dividing a sum of confidence measures at a pixel location of the pixel by a sum of confidence measures of the plurality of contributors.
5. The method of claim 1 , wherein the generating of the additional image channel includes, for a pixel in the image, dividing a sum of confidence measures at a pixel location of the pixel by a number of contributors.
6. The method of claim 1 , wherein the generating of the additional image channel includes:
performing edge smoothing on a plurality of bounding boxes included in the plurality of annotation data sets; and
determining, for a pixel in the image, a sum of associated values with respect to one or more edge-smoothed bounding boxes.
7. The method of claim 1 , wherein the additional image channel is generated for a plurality of objects in the image.
8. The method of claim 1 , wherein the additional image channel is generated for a single object in the image.
9. The method of claim 1 , wherein the applying of the object detection model includes:
generating a feature map based at least in part on the image and the additional image channel.
10. The method of claim 9 , further comprising:
applying a plurality of anchors to a feature in the feature map; and
regressing based on the plurality of anchors to determine a best fitting bounding box.
11. The method of claim 10 , further comprising classifying the feature.
12. A system for image annotation, comprising:
one or more processors configured to:
access an image and a plurality of annotation data sets for the image, wherein the plurality of annotation data sets are made by a plurality of contributors, and the image has a plurality of original image channels;
aggregate the plurality of annotation data sets to obtain an aggregated annotation data set for the image, including to:
generate an additional image channel based at least in part on weight averages of confidence measures of the plurality of contributors; wherein a confidence measure of a contributor indicates an accuracy of the contributor in making annotations or how confident the contributor is at making an annotation; and
apply an object detection model to at least a part of the plurality of original image channels and at least a part of the additional image channel to generate the aggregated annotation data set; and
output the aggregated annotation data set; and
one or more memories coupled to the one or more processors and configured to provide the one or more processors with instructions.
13. The system of claim 12 , wherein to generate the additional image channel includes to determine, for a pixel in the image, a sum of associated values with respect to one or more bounding boxes.
14. The system of claim 13 , wherein the associated values with respect to the one or more bounding boxes are associated with the confidence measures of the plurality of contributors.
15. The system of claim 12 , wherein to generate the additional image channel includes, for a pixel in the image, to divide a sum of confidence measures at a pixel location of the pixel by a sum of confidence measures of the plurality of contributors.
16. The system of claim 12 , wherein to generate the additional image channel includes, for a pixel in the image, to divide a sum of confidence measures at a pixel location of the pixel by a number of contributors.
17. The system of claim 12 , wherein to generate the additional image channel includes to:
perform edge smoothing on a plurality of bounding boxes included in the plurality of annotation data sets; and
determine, for a pixel in the image, a sum of associated values with respect to one or more edge-smoothed bounding boxes.
18. The system of claim 12 , wherein the additional image channel is generated for a plurality of objects in the image.
19. The system of claim 12 , wherein the additional image channel is generated for a single object in the image.
20. The system of claim 12 , wherein to apply the object detection model includes to:
generate a feature map based at least in part on the image and the additional image channel.
21. The system of claim 20 , wherein the one or more processors are further configured to:
apply a plurality of anchors to a feature in the feature map; and
regress based on the plurality of anchors to determine a best fitting bounding box.
22. The system of claim 21 , wherein the one or more processors are further configured to classify the feature.
23. A computer program product for image annotation, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:
accessing an image and a plurality of annotation data sets for the image, wherein the plurality of annotation data sets are made by a plurality of contributors, and the image has a plurality of original image channels;
aggregating the plurality of annotation data sets to obtain an aggregated annotation data set for the image, including:
generating an additional image channel based at least in part on weight averages of confidence measures of the plurality of contributors; wherein a confidence measure of a contributor indicates an accuracy of the contributor in making annotations or how confident the contributor is at making an annotation; and
applying an object detection model to at least a part of the plurality of original image channels and at least a part of the additional image channel to generate the aggregated annotation data set; and
outputting the aggregated annotation data set.