SYSTEMS AND METHODS FOR FEW SHOT OBJECT DETECTION
A system may be configured to detect an unseen object. Some embodiments may: train a machine learning (ML) model, with training data and with both a positive-support content item and a negative-support content item; and predict, via the trained ML model, presence, within a region, of an object in a newly-obtained content item. The object may (i) not have previously been used to train the ML model and (ii) be among a background and a candidate object present in the newly-obtained content item.
1 . A method for detecting an unseen object, the method comprising:
obtaining a positive-support content item and a plurality of training data, each datum comprising one or more content items;
training a machine learning model with the training data and the positive-support content item; and
predicting, via the trained machine learning model, presence, within a region, of an object in a newly-obtained content item,
wherein the object (i) has not previously been used to train the model and (ii) is among a background and a candidate object present in the newly-obtained content item.
2 . The method of claim 1 , wherein the prediction is based on the region matching a region of the positive-support content item.
3 . The method of claim 1 , wherein the model is further trained with a negative-support content item.
4 . The method of claim 3 , wherein the positive-support content item is previously provided by a user of a system performing the object detection, and
wherein the negative support content item comprises objects the presence of which the model is not intended to predict.
5 . The method of claim 3 , wherein weights of a backbone neural network are shared, when the model is (i) training with the positive-support and negative-support content items and (ii) obtaining the newly-obtained content items.
6 . The method of claim 1 , wherein the prediction is made without having to retune any weights of the model subsequent to the training.
7 . The method of claim 2 , wherein the object is depicted, in the newly-obtained content item, differently from the object's depiction, in the positive-support content item.
8 . The method of claim 7 , wherein the different depictions of the object (i) comprise a different background, (ii) comprise different instances of a same type of the object, and/or (ii) are captured at different times.
9 . The method of claim 2 , wherein the prediction comprises:
first-detecting a region of each of at least the object and the candidate object;
second-detecting, from among the first-detected regions, an object having a regional similarity score that satisfies a criterion with respect to the positive-support content item; and
displaying, via a user interface, only the region in which the second-detected object is present.
10 . The method of claim 9 , wherein the second-detection is performed by determining a regional similarity score with respect to each of all candidate objects and the positive-support content item.
11 . The method of claim 10 , wherein the model comprises a faster recurrent convolutional neural network (R-CNN) to which are coupled a pair of region of interest (ROI) pooling layers and a similarity model that computes the similarity scores.
12 . The method of claim 1 , wherein the newly-obtained content item comprises a series of time-sequential images or video.
13 . The method of claim 3 , wherein the object satisfies a uniqueness or rarity criterion that is higher than a uniqueness or rarity criterion of objects in the negative-support content item.
14 . A method for detecting an unseen object, the method comprising:
first-detecting, via a first machine learning model from among each of a plurality of images, a region of each of at least the unseen object and a candidate object;
second-detecting, via a second machine learning model operably coupled to the first machine learning model, and from among the first-detected regions, an object having a regional similarity score that satisfies a criterion with respect to a positive-support content item; and
displaying, via a user interface in the plurality of images, only bounds of the region in which the second-detected object is present.
15 . The method of claim 14 , wherein the second-detection is performed by determining a regional similarity score with respect to each of all candidate objects and the positive-support content item.
16 . The method of claim 14 , wherein the first model is trained with training data, a positive-support image, and a negative-support image.
17 . The method of claim 16 , wherein the positive-support image is previously provided by a user of a system performing the object detection, and
wherein the negative support image comprises objects the presence of which the second model is not intended to predict.
18 . The method of claim 16 , wherein weights of a backbone neural network are shared, when (i) training with the positive-support and negative-support images and (ii) newly-obtaining the plurality of images.
19 . A method, comprising:
obtaining a first positive-support image and a second positive-support image;
obtaining a real-time video stream;
training a machine learning model with training data and the images;
tagging the stream with the first positive-support image such that the model predicts presence of a first object, while the video stream is being played; and
subsequently tagging, in real-time, the stream with the second positive-support image such that the model predicts presence of a second object,
wherein neither the first nor second object has been used to perform the training, and wherein the first and second objects are of a different type.
20 . The method of claim 19 , wherein the method further comprises obtaining a negative-support image such that the model is further trained with the negative-support image, and
wherein each of the first and second objects satisfies a uniqueness or rarity criterion that is higher than a uniqueness or rarity criterion of objects in the negative-support image.