Pixel-level based micro-feature extraction
Techniques are disclosed for extracting micro-features at a pixel-level based on characteristics of one or more images. Importantly, the extraction is unsupervised, i.e., performed independent of any training data that defines particular objects, allowing a behavior-recognition system to forgo a training phase and for object classification to proceed without being constrained by specific object definitions. A micro-feature extractor that does not require training data is adaptive and self-trains while performing the extraction. The extracted micro-features are represented as a micro-feature vector that may be input to a micro-classifier which groups objects into object type clusters based on the micro-feature vectors.
1. A method, comprising:
identifying, via a processor, a plurality of foreground objects depicted in a sequence of video frames;
for each foreground object from the plurality of foreground objects, deriving feature data for that foreground object from each video frame from the sequence of video frames that depicts the foreground object;
generating, via the processor and based on the derived feature data of a first foreground object from the plurality of foreground objects, an object type model;
correlating, via the processor, the derived feature data of a second foreground object from the plurality of foreground objects with the object type model by mapping the derived feature data of the second foreground object to an adaptive resonance theory (ART) network; and
in response to the correlating the derived feature data for the second foreground object with the object type model, assigning an object type identifier to the second foreground object to indicate that the second foreground object is an instance of an object type associated with the object type model.
2. The method of claim 1 , wherein the object type model includes the ART network.
3. The method of claim 1 , wherein the generating the object type model includes generating a cluster in the ART network based on the derived feature data of the first foreground object.
4. The method of claim 1 , wherein the mapping the derived feature data of the second foreground object to the ART network includes mapping the derived feature data of the second foreground object to a cluster in the ART network.
5. The method of claim 1 , further comprising updating the object type model based on the derived feature data of the second foreground object.
6. The method of claim 1 , wherein the derived feature data for each foreground object from the plurality of foreground objects includes static data characterizing that foreground object in the sequence of video frames that depicts the foreground object.
7. The method of claim 1 , wherein the derived feature data for each foreground object from the plurality of foreground objects includes kinematic data characterizing that foreground object in the sequence of video frames that depicts the foreground object.
8. A non-transitory, processor-readable medium storing instructions that, when executed by a processor, cause the processor to:
for each foreground object from a plurality of foreground objects depicted in a sequence of video frames, derive feature data for that foreground object from each video frame from the sequence of video frames that depicts the foreground object;
generate, based on the derived feature data of a first foreground object from the plurality of foreground objects, an object type model;
map the derived feature data of a second foreground object from the plurality of foreground objects to an adaptive resonance theory (ART) network; and
in response to the correlating the derived feature data for the second foreground object with the object type model, assign an object type identifier to the second foreground object to indicate that the second foreground object is an instance of an object type associated with the object type model.
9. The non-transitory, processor-readable medium of claim 8 , wherein the object type model includes the ART network.
10. The non-transitory, processor-readable medium of claim 8 , wherein the instructions to generate the object type model include instructions to generate a cluster in the ART network based on the derived feature data of the first foreground object.
11. The non-transitory, processor-readable medium of claim 8 , wherein the instructions to map the derived feature data of the second foreground object to the ART network include instructions to map the derived feature data of the second foreground object to a cluster in the ART network.
12. The non-transitory, processor-readable medium of claim 8 , further storing instructions to cause the processor to update the object type model based on the derived feature data of the second foreground object.
13. The non-transitory, processor-readable medium of claim 8 , wherein the derived feature data for each foreground object from the plurality of foreground objects includes static data characterizing that foreground object in the sequence of video frames that depicts the foreground object.
14. The non-transitory, processor-readable medium of claim 8 , wherein the derived feature data for each foreground object from the plurality of foreground objects includes kinematic data characterizing that foreground object in the sequence of video frames that depicts the foreground object.
15. A system, comprising:
a processor; and
a memory operably coupled to the processor and storing instructions that, when executed by the processor, cause the processor to:
for each foreground object from a plurality of foreground objects depicted in a sequence of video frames, derive feature data for that foreground object from each video frame from the sequence of video frames that depicts the foreground object;
generate, based on the derived feature data of a first foreground object from the plurality of foreground objects, an object type model that includes an adaptive resonance theory (ART) network;
correlate the derived feature data of a second foreground object from the plurality of foreground objects with the object type model by mapping the derived feature data to object type model; and
in response to the correlating the derived feature data for the second foreground object with the object type model, assign an object type identifier to the second foreground object to indicate that the second foreground object is an instance of an object type associated with the object type model.
16. The system of claim 15 , wherein the memory further stores instructions to cause the processor to output, to a machine learning engine, kinematic data based on the object type identifier, the kinematic data describing a behavior of the second foreground object.
17. The system of claim 15 , wherein the instructions to generate the object type model include instructions to generate a cluster in the ART network based on the derived feature data of the first foreground object.
18. The system of claim 15 , wherein the instructions to correlate the derived feature data of the second foreground object with the object type model include instructions to map the derived feature data of the second foreground object to a cluster in the object type model.
19. The system of claim 15 , wherein the memory further stores instructions to cause the processor to update the object type model based on the derived feature data of the second foreground object.
20. The system of claim 15 , wherein the derived feature data for each foreground object from the plurality of foreground objects depicted in the sequence of video frames includes at least one of static data or kinematic data characterizing that foreground object in the sequence of video frames that depicts the foreground object.