Computer vision model drawing interface
For updating a computer vision model, a method converts a user input drawing including a user annotation of a first image in a drawing format to a training format image in a training format for a computer vision model. The method generates a training-representation drawing from the training format image. The training-representation drawing includes an image inference for the first image. The method receives user feedback for the training-representation drawing in the drawing format. The method updates the computer vision model based on the user feedback. The method generates an image inference for a second image based on the updated computer vision model and generates model-health metrics, agreement-metrics, and a sortable image index to explain image inferences with respect to guided user annotation of the second image. The method caches partial results from the image-inferences to afford quicker updating of computer vision models, affording more iterative model-development than ad-hoc model-evaluation.
1. A method comprising:
converting, by use of processor, a user input drawing comprising a user annotation of a first image in a drawing format to a training format image in a training format for a computer vision model;
generating, with the computer vision model, a training-representation drawing from the training format image, the training-representation drawing comprising an image inference for the first image;
receiving user feedback for the training-representation drawing in the drawing format;
updating the computer vision model based on the user feedback; and
generating an image inference for a second image based on the updated computer vision model.
2. The method of claim 1 , wherein the image inference comprises a pixel map agreement metric that highlights commonly marked and differently marked pixels in the user annotation and image annotations, a pixel percentage metric that compares the commonly marked and differently marked pixels in the user annotation and image annotations, and a comparison heatmap metric that shows the sums of the commonly marked and differently marked pixels over regions.
3. The method of claim 1 , wherein the method further comprises:
receiving training data;
pre-training the computer vision model;
receiving the user input drawing;
sorting at least two training-representation drawings;
presenting the training-representation drawing; and
modifying a machine component based on the image inference for the second image.
4. The method of claim 3 , wherein the computer vision model is pre-training using transfer learning and/or meta learning.
5. The method of claim 1 , wherein the method further comprises:
comparing at least two sets of training-representation drawings, user input drawings, and/or corresponding feedback information;
identifying a user consistency from the comparison;
identifying a model health from the comparison;
reconciling conflicts between the at least two sets of training-representation drawings, user input drawings, and/or corresponding feedback information; and
improving the model health to a target model health range.
6. The method of claim 5 , wherein the model health is presented as a user interface that indicates image frequency, inference frequency, label frequency, and/or overall health.
7. The method of claim 6 , wherein the overall health is presented as a color gradient background.
8. The method of claim 1 , wherein the drawing format is selected from the group consisting of a bounding box, a mask, and selected pixels.
9. The method of claim 1 , the method further comprising:
recording a process history comprising process information, images, and corresponding image inferences;
appending the process history to the training data; and
updating the computer vision model with the training data.
10. The method of claim 1 , the method further comprising:
identifying a process condition;
presenting a second image and process information corresponding to the process condition; and
updating the computer vision model from a second user annotation of the second image.
11. The method of claim 1 , the training-representation drawing further comprising a saliency map indicating areas of the first image that influenced the model inference.
12. The method of claim 1 , the training-representation drawing further comprising feedback information comprising a coverage agreement, an intersection over union (IoU) coverage, a saliency coverage, a ground truth coverage, a shared focus area, a divergent focus area, a confusion area, and a suggestion.
13. The method of claim 1 , wherein the user feedback comprises an agreement indication, an annotation bounding box, annotation pixels, annotation colors, and/or annotation layers.
14. The method of claim 1 , wherein the computer vision model is updated with training data comprising images and corresponding image inferences and user feedback.
15. The method of claim 1 , wherein the computer vision model is updated with training data comprising images and corresponding image inferences that incorporate user feedback.
16. An apparatus comprising:
a processor executing code stored on a memory to perform:
converting a user input drawing comprising a user annotation of a first image in a drawing format to a training format image in a training format for a computer vision model;
generating, with the computer vision model, a training-representation drawing from the training format image, the training-representation drawing comprising an image inference for the first image;
receiving user feedback for the training-representation drawing in the drawing format;
updating the computer vision model based on the user feedback; and
generating an image inference for a second image based on the updated computer vision model.
17. The apparatus of claim 16 , wherein the image inference comprises a pixel map agreement metric that highlights commonly marked and differently marked pixels in the user annotation and image annotations, a pixel percentage metric that compares the commonly marked and differently marked pixels in the user annotation and image annotations, and a comparison heatmap metric that shows the sums of the commonly marked and differently marked pixels over regions.
18. The apparatus of claim 16 , wherein the processor further performs:
receiving training data;
pre-training the computer vision model;
receiving the user input drawing;
sorting at least two training-representation drawings;
presenting the training-representation drawing; and
modifying a machine component based on the image inference for the second image.
19. The apparatus of claim 18 , wherein the computer vision model is pre-training using transfer learning and/or meta learning.
20. A computer program product, the computer program product comprising a non-transitory computer readable storage medium having program code embodied therein, the program code readable/executable by a processor to:
converting a user input drawing comprising a user annotation of a first image in a drawing format to a training format image in a training format for a computer vision model;
generating, with the computer vision model, a training-representation drawing from the training format image, the training-representation drawing comprising an image inference for the first image;
receiving user feedback for the training-representation drawing in the drawing format;
updating the computer vision model based on the user feedback; and
generating an image inference for a second image based on the updated computer vision model.