Computer vision image feature identification via multi-label few-shot model
A technique making use of a few-shot model to determine graphical features present in an image based on a small set of examples with known graphical features. Where a support set including a number of images that each have a known combination of graphical features, the image recognition can identify unknown combinations of those graphical features in any number of query images. In an embodiment of the present disclosure examples of a filled-out form are used to interpret any number of additional filled out versions of the form.
1. A method comprising:
training a few-shot model with a few-shot data set, the few-shot data set including a number of images, each image of the number of images having a combination of graphic features, wherein the few-shot data set does not include every combination of the graphic features;
generating a matrix of graphic features of the few-shot data set;
receiving a query image by the few-shot model, the query image including a query combination of graphic features using graphic features found within the few-shot data set and represented by the matrix of graphic features; and
identifying graphic features present in the query image via the few-shot model based on the matrix of graphic features.
2. The method of claim 1 , further comprising:
providing the few-shot model with a significant training data set that is unrelated to the specific graphic features of the query image.
3. The method of claim 1 , further comprising:
generating a query matrix of graphic features of the query image; and
wherein said identifying graphic features is further based on the query matrix.
4. The method of claim 1 , wherein each image of the few-shot data set and the query image are each a representation of a filled-out form document, the filled-out form documents adhering to a same form template, and the query combination of graphical features are check boxes positioned at predetermined locations on the form template.
5. The method of claim 1 , wherein the query image includes an unknown combination of the graphical features that is not present within the few-shot data set.
6. The method of claim 1 , further comprising:
identifying a set of known graphical features of the few-shot data set via application of a computer vision model to each of the number of images.
7. The method of claim 1 , further comprising:
generating few-shot vectors based on each the number of images of the few-shot data set; and
generating a query vector based on the query image.
8. The method of claim 7 , further comprising:
concatenating the query vector to each of the few-shot vectors resulting in combined vectors; and
submitting each of the combined vectors to the few-shot model as input.
9. The method of claim 3 , wherein said identifying is further based on a pairwise comparison that makes use of the query matrix and the matrix of graphic features.
10. A method of computer vision analysis of images that identifies graphical features in images via a few-shot model comprising:
training a first machine learning model via a set of images, the set of images each including graphical features, wherein the set of images is a few-shot;
for each image of the set of images, generating a respective vector representative of each of the set of images;
inputting a query image to the trained machine learning model, the query image including a query combination of graphic features using graphic features found within the few-shot and represented by the matrix of graphic features;
generating a query vector representative of the query image;
performing a pairwise comparison based on the query vector and the respective vectors, wherein the pairwise comparison indicates flail graphical features in common between the query image and each of the set of images respectively; and
deriving the graphical features present in the query image based on said pairwise comparison.
11. The method of claim 10 , further comprising:
providing the first model with a significant training data set that is unrelated to the graphical features present in the query image.
12. The method of claim 10 , further comprising:
concatenating the query vector to each of the respective vectors resulting in combined vectors; and
submitting each of the combined vectors to the first model as input.
13. The method of claim 10 , wherein each image of the set of images and the query image are each a representation of a filled-out form document, the filled-out form documents adhering to a same form template, and the graphical features are check boxes positioned at predetermined locations on the form template.
14. The method of claim 10 , wherein the set of images includes at least one instance of each of the graphical features and the query image includes a combination of the graphical features that is not present within the set of images.
15. A system of computer vision analysis of images that identifies graphical features in images via a few-shot model comprising:
a graphic features model configured to generate a set of vectors representative of graphical features in each of a few-shot set of images and a query image, where the graphical features present in the set of images are known graphical features to the graphical features model and the graphic features present in the query image are unknown to the graphics features model;
a processor; and
a memory including instructions configured to cause the processor to:
training the graphic features model with a few-shot set of images, the few-shot set of images, each image of the set of images having a combination of graphic features, wherein the few-shot set of images does not include every combination of the graphic features;
receiving the query image by the graphic features model, the query image including a combination of graphic features using known graphic features found in at least one image of the few-shot set of images; and
identify graphic features present in the query image via the graphic features model based on the known graphical features.
16. The system of claim 15 , wherein each image of the few-shot set of images and the query image are each a representation of a filled-out form document, the filled-out form documents adhering to a same form template, and the known graphical features are check boxes positioned at predetermined locations on the form template.
17. The system of claim 15 , wherein the few-shot set of images includes at least one instance of each of the graphical features and the query image includes an unknown combination of the graphical features that is not present within the set of images.
18. The system of claim 15 , the instructions further comprising:
generating few-shot vectors based on each the few-shot set of images; and
generating a query vector based on the query image.
19. The system of claim 18 , the instructions further comprising:
concatenating the query vector to each of the few-shot vectors resulting in combined vectors; and
submitting each of the combined vectors to the graphic features model as input.
20. The system of claim 15 , further comprising:
a computer vision model configured to characterize the graphical features of the few-shot set of images.