Data collection and classifier training in edge video devices
A digital video camera architecture for updating an object identification and tracking model deployed with the camera is disclosed. The invention comprises optics, a processor, a memory, and an artificial intelligence logic which may further comprise artificial neural networks. The architecture may identify objects according to the confidence threshold of a model. The confidence threshold may be monitored over time, and the model may be updated if the confidence threshold drops below an acceptable level. The data for retraining is ideally generated substantially internal to the camera. A classifier is generated to process the entire field data set stored on the camera to create a field data subset also stored on the camera. The field data subset may be run through the model to generate cases that may be used in further monitoring, training, and updating of the model. Classifiers may also be generated for images in different domains (e.g., lighting, weather, surveillance area, indoor, outdoor, urban, rural, etc.). These classifiers can be used to train the model to accurately identify objects and features independent of the domain of origin of the image being evaluated.
1 . A device, comprising:
an artificial intelligence logic configured to:
evaluate a source domain validation data set with a first detector,
use annotations of an output of the first detector to generate a source domain surrogate data set,
train a source domain classifier with the source domain surrogate data set,
evaluate a source domain entire field data set with the source domain classifier to generate a source domain field data subset,
evaluate the source domain field data subset with the first detector to generate a source domain final field data subset,
update the source domain validation data set from the source domain final field data subset, and
train, using the source domain classifier, a target domain classifier to:
minimize an error function at an output of a label classification head of the target domain classifier, and
maximize the error function at an output of a domain classification head of the target domain classifier to filter features that are unique to the source domain classifier.
2 . The device of claim 1 , wherein:
the target domain classifier comprises a feature extractor derived from a first portion of the source domain classifier;
the label classification head is derived from a second portion of the source domain classifier; and
the domain classification head is coupled in parallel to the label classification head.
3 . The device of claim 2 , wherein:
the artificial intelligence logic is further configured to:
extract first features from the source domain validation data set using the feature extractor,
extract second features from a target domain validation data set using the feature extractor,
run the first features through both the label classification head and the domain classification head, and
run the second features only through the domain classification head.
4 . The device of claim 3 , wherein:
the artificial intelligence logic is further configured to:
apply positive feedback to the output of the label classification head, and
apply negative feedback to the output of the domain classification head.
5 . The device of claim 4 , wherein the negative feedback is adversarial training.
6 . The device of claim 4 , wherein the positive feedback is backpropagation to reinforce features similar to the source domain classifier.
7 . The device of claim 4 , wherein:
the artificial intelligence logic is further configured to:
train the target domain classifier to:
label objects in images with substantially the same accuracy at the output of the label classification head with images from either the source domain validation data set or the target domain validation data set,
update a main validation data set from the source domain validation data set,
update the main validation data set from the target domain validation data set, and
train a second detector using the updated main validation data set.
8 . The device of claim 1 , further comprising:
an optics module.
9 . A system, comprising:
a computational system configured to:
evaluate a source domain validation data set with a first detector,
use annotation of an output of the first detector to generate a source domain surrogate data set, and
train a source domain classifier with the source domain surrogate data set; and
a device, comprising:
an artificial intelligence logic configured to:
run a source domain entire field data set through the source domain classifier to generate a source domain field data subset,
evaluate the source domain field data subset with the first detector to generate a source domain final field data subset,
update the source domain validation data set from the source domain final field data subset, and
train, using the source domain classifier, a target domain classifier to:
minimize an error function at an output of a label classification head of the target domain classifier, and
maximize the error function at an output of a domain classification head of the target domain classifier to filter features that are unique to the source domain classifier.
10 . The system of claim 9 , further comprising:
an optics module.
11 . The system of claim 9 , wherein:
the target domain classifier comprises a feature extractor derived from a first portion of the source domain classifier;
the label classification head is derived from a second portion of the source domain classifier; and
the domain classification head is coupled in parallel to the label classification head.
12 . The system of claim 11 , wherein:
the artificial intelligence logic is further configured to:
extract first features from the source domain validation data set using the feature extractor,
extract second features from a target domain validation data set using the feature extractor,
run the first features through both the label classification head and the domain classification head, and
run the second features only through the domain classification head.
13 . The system of claim 12 , wherein:
the artificial intelligence logic is further configured to:
train the target domain classifier to:
label objects in images with substantially the same accuracy at the output of the label classification head with images from either the source domain validation data set or the target domain validation data set,
update a main validation data set from the source domain validation data set,
update the main validation data set from the target domain validation data set, and
train a second detector using the updated main validation data set.
14 . The system of claim 12 , wherein:
the artificial intelligence logic is further configured to:
apply positive feedback to the output of the label classification head, and
apply negative feedback to the output of the domain classification head.
15 . The system of claim 14 , wherein the positive feedback is backpropagation to reinforce features similar to the source domain classifier.
16 . The system of claim 14 , wherein the negative feedback is adversarial training.
17 . A system, comprising:
means for evaluating a source domain validation data set with a first detector;
means for using annotations of an output of the first detector to generate a source domain surrogate data set;
means for training a source domain classifier with the source domain surrogate data set;
means for running a source domain entire field data set through the source domain classifier to generate a source domain field data subset;
means for evaluating the source domain field data subset with the first detector to generate a source domain final field data subset;
means for updating the source domain validation data set from the source domain final field data subset; and
means for training, using the source domain classifier, a target domain classifier to:
minimize an error function at an output of a label classification head of the target domain classifier, and
maximize the error function at an output of a domain classification head of the target domain classifier to filter features that are unique to the source domain classifier, wherein;
the target domain classifier comprises a feature extractor derived from a first portion of the source domain classifier,
the label classification head is derived from a second portion of the source domain classifier, and
the domain classification head is coupled in parallel to the label classification head.
18 . The system of claim 17 , further comprising:
means for extracting first features from the source domain validation data set using the feature extractor;
means for extracting second features from a target domain validation data set using the feature extractor;
means for running the first features through both the label classification head and the domain classification head;
means for running the second features only through the domain classification head;
means for applying positive feedback to the output of the label classification head; and
means for applying negative feedback to the output of the domain classification head.
19 . The system of claim 18 , further comprising:
means for updating a main validation data set from the source domain validation data set;
means for updating the main validation data set from the target domain validation data set; and
means for training a second detector using the updated main validation data set.
20 . The system of claim 18 , further comprising:
a digital video camera.