Method and system for training a machine learning model with a subclass of one or more predefined classes of visual objects
A system and a method for training a machine learning model with a subclass of one or more predefined classes of visual objects obtained from a video, are provided herein. The method includes: presenting to a human operator, over an electronic display, a plurality of visual objects obtained from the video, wherein the plurality of visual objects belongs to one or more predefined classes of visual objects; receiving from the human operator, a selection of some of the plurality of visual objects, wherein the selection is directed at visual objects belonging to at least one subclass of the one or more predefined classes; and training a machine learning model, using a computer processor, to distinguish between visual objects belonging to the at least one subclass and visual objects belonging to the predefined one or more classes but not belonging to the at least one subclass, based on said selection.
1 . A method of training a machine learning model with a subclass of one or more predefined classes of visual objects obtained from one or more videos, the method comprising:
presenting to a human operator, over an electronic display, a plurality of presented visual objects obtained from the one or more videos, wherein the plurality of presented visual objects belongs to one predefined class of visual objects;
receiving from the human operator, over a user interface associated with the electronic display, selected visual objects being a selection of some of the plurality of the presented visual objects, wherein the selection is directed at visual objects belonging to at least one subclass of the one predefined class;
creating, by a computer processor, additional samples of the selected visual objects belonging to at least one subclass of the one predefined class, by tracking the selected visual objects across video frames of the one or more videos and extracting corresponding tracked thumbnail images of the selected visual objects from different frames;
carrying out supervised training of a classifier, using the computer processor, the selected visual objects and the additional samples of the selected visual objects, together with non-selected visual objects from the plurality of presented visual objects as negative examples, to distinguish between the selected visual objects and the presented visual objects which were not selected; and
applying the classifier as a subclass filter in an object-classification pipeline for classifying visual objects in other videos, as belonging to the selected at least one subclass.
2 . The method according to claim 1 , further comprising: determining, using the computer processor and based on the trained model, whether or not a newly obtained visual object belongs to the at least one subclass.
3 . The method according to claim 1 , wherein the at least one subclass comprises at least a first subclass and a second sub class wherein the selection and the training is carried out for the first subclass and then repeated with the second subclass.
4 . The method according to claim 1 , wherein the at least one subclass comprises at least a first subclass and a second sub class wherein the selection is carried out for the first and the second subclasses and then the training is carried out for the first and the second subclasses simultaneously.
5 . The method according to claim 1 , wherein the one or more predefined classes of the visual objects comprises a single predefined class of the visual objects.
6 . The method according to claim 1 , further comprising automatically suggesting to the human operator which class of the one or more predefined classes of the visual objects may best benefit from having subclasses.
7 . A system for training a machine learning model with a subclass of one or more predefined classes of visual objects obtained from one or more videos, the system comprising:
a computer memory configured to store one or more input videos comprising visual objects;
a classifier implemented by a computer processor configured to classify the visual objects from the input videos into a plurality of predefined classes;
an electronic display configured to present to a human operator, a plurality of presented visual objects obtained from the one or more videos, belonging to one of the one predefined class requested by the human operator;
a user interface associated with the electronic display, configured to receive from the human operator, selected visual objects being a selection of some of the plurality of the presented visual objects, wherein the selection is directed at visual objects belonging to at least one subclass of the one predefined class and
a machine learning module, implemented by the computer processor, configured to create additional samples of the selected visual objects by tracking said selected visual objects across video frames of the one or more videos and to carry out supervised training of a classifier, using the selected visual objects, the additional samples of the selected visual objects, and non-selected visual objects as negative examples, to distinguish between the selected visual objects and the presented visual objects which were not selected,
wherein the computer processor applies the classifier as a subclass filter for classifying visual objects in other videos, as belonging to the selected at least one subclass.
8 . The system according to claim 7 , wherein the computer processor is further configured to determine, based on the trained model, whether or not a newly obtained visual object belongs to the at least one subclass.
9 . The system according to claim 7 , wherein the at least one subclass comprises at least a first subclass and a second sub class wherein the selection and the training is carried out for the first subclass and then repeated with the second subclass.
10 . The system according to claim 7 , wherein the at least one subclass comprises at least a first subclass and a second sub class wherein the selection is carried out for the first and the second subclasses and then the training is carried out for the first and the second subclasses simultaneously.
11 . The system according to claim 7 , wherein the one or more predefined classes of the visual objects comprises a single predefined class of the visual objects.
12 . The system according to claim 7 , wherein the computer processor is further configured to automatically suggest to the human operator which class of the one or more predefined classes of the visual objects may best benefit from having subclasses.
13 . A non-transitory computer readable medium for training a machine learning model with a subclass of one or more predefined classes of visual objects obtained from one or more videos, the computer readable medium comprising a set of instructions that, when executed, cause at least one computer processor to:
classify the visual objects from the one or more videos into a plurality of predefined classes;
present to a human operator, over an electronic display, a plurality of presented visual objects obtained from the one or more videos, belonging to the one predefined class requested by the human operator;
receive from the human operator, over a user interface associated with the electronic display, a selection of some of the plurality of visual objects, wherein the selection is directed at visual objects belonging to at least one subclass of the one predefined class;
create additional samples of the selected visual objects by tracking said selected visual objects across video frames of the one or more videos and extracting corresponding tracked thumbnail images of the selected visual objects from different frames;
carry out supervised training of a classifier using the selected visual objects, the additional samples of the selected visual objects, and non-selected visual objects as negative examples, to distinguish between the selected visual objects and the presented visual objects which were not selected; and
apply the classifier as a subclass filter for classifying visual objects in other videos, as belonging to the selected at least one subclass.
14 . The non-transitory computer readable medium according to claim 13 , further comprises determining, using the computer processor and based on the trained model, whether or not a newly obtained visual object belongs to the at least one subclass.
15 . The non-transitory computer readable medium according to claim 13 , wherein the at least one subclass comprises at least a first subclass and a second sub class wherein the selection and the training is carried out for the first subclass and then repeated with the second subclass.
16 . The non-transitory computer readable medium according to claim 13 , wherein the at least one subclass comprises at least a first subclass and a second sub class wherein the selection is carried out for the first and the second subclasses and then the training is carried out for the first and the second subclasses simultaneously.
17 . The non-transitory computer readable medium according to claim 13 , wherein the one or more predefined classes of the visual objects comprises a single predefined class of the visual objects.
18 . The non-transitory computer readable medium according to claim 13 , wherein the computer processor is further configured to automatically suggest to the human operator which class of the one or more predefined classes of the visual objects may best benefit from having subclasses.