Pixel-level based micro-feature extraction
Techniques are disclosed for extracting micro-features at a pixel-level based on characteristics of one or more images. Importantly, the extraction is unsupervised, i.e., performed independent of any training data that defines particularly objects, allowing a behavior-recognition system to forgo a training phase and for object classification to proceed without being constrained by specify object definitions. A micro-feature extractor that does not require training data is adaptive and self-trains while performing the extraction. The extracted micro-features are represented as a micro-feature vector that may be input to a micro-classifier which groups object into object type clusters based on the micro-feature vectors.
1. A system, comprising:
at least one processor; and
a memory in communication with the at least one processor and storing instructions for the at least one processor to:
generate a micro-feature vector based on at least one of the kinematics of an object in a video scene or an appearance of the object in the video scene;
update a neural network based at least in part on the micro-feature vector to produce an updated neural network, the neural network including a plurality of clusters, each cluster from the plurality of clusters representing an object type,
classify the object as a first object type based on the updated neural network, the first object type being represented by a first cluster from the plurality of clusters, and
track the object in the video scene based on the micro-feature vector and the first object type.
2. The system of claim 1 , wherein the instructions for the at least one processor to update the neural network include instructions for the at least one processor to:
evaluate a rank for each cluster from the plurality of clusters relative to the micro-feature vector, and
evaluate each cluster from the plurality of clusters to determine if the object maps to at least one cluster from the plurality of clusters based on the rank for each cluster.
3. The system of claim 1 , wherein the instructions for the at least one processor to update the neural network include instructions for the at least one processor to generate the first cluster representing the first object type.
4. The system of claim 1 , wherein the instructions for the at least one processor to update the neural network include instructions for the at least one processor to update the first cluster based on the micro-feature vector.
5. The system of claim 1 , wherein the instructions include instructions for the at least one processor to reinforce a machine learning pattern as events occur and re-occur within the video scene.
6. A processor-implemented method, comprising:
generating a micro-feature vector based on at least one of kinematics of an object in a video scene or an appearance of the object in the video scene;
updating a neural network based on the micro-feature vector to produce an updated neural network, the neural network including a plurality of clusters, each cluster from the plurality of clusters representing an object type;
classifying the object as a first object type based at least in part on the updated neural network, the first object type being associated with a first cluster from the plurality of clusters; and
tracking the object based on the micro-feature vector and the first object type.
7. The processor-implemented method of claim 6 , wherein updating the neural network includes generating the first cluster.
8. The processor-implemented method of claim 6 , wherein updating the neural network includes updating the first cluster based on the micro-feature vector.
9. The processor-implemented method of claim 6 , comprising generating a plurality of micro-feature vectors, the plurality of micro-feature vectors including the micro-feature vector, the method further comprising:
generating the plurality of clusters based on the plurality of micro-feature vectors.
10. The processor-implemented method of claim 6 , wherein at least one micro-feature value associated with the micro-feature vector includes a verticality of the foreground patch.
11. The processor-implemented method of claim 6 , further comprising evaluating a rank for each cluster from the plurality of clusters relative to the micro-feature vector.
12. The processor-implemented method of claim 6 , further comprising evaluating each cluster from the plurality of clusters to determine whether the object maps to at least one cluster from the plurality of clusters based on the ranking for each cluster.
13. The processor-implemented method of claim 6 , further comprising reinforcing a machine learning pattern as events occur and re-occur within the video scene.
14. A non-transitory processor-readable storage medium storing processor-executable instructions to:
generate a micro-feature vector based on at least one of kinematics of an object in a video scene or an appearance of the object in the video scene;
update a neural network based on the micro-feature vector to produce an updated neural network, the neural network including a plurality of clusters, each cluster from the plurality of clusters representing an object type;
classify the object as a first object type based at least in part on the updated neural network, the first object type being associated with a first cluster from the plurality of clusters; and
track the object based on at least one of the micro-feature vector or the first object type.
15. The non-transitory processor-readable storage medium of claim 14 , wherein the instructions to update the neural network include instructions to generate the first cluster.
16. The non-transitory processor-readable storage medium of claim 14 , wherein the instructions to update the neural network include instructions to update the first cluster based on the micro-feature vector.
17. The non-transitory processor-readable storage medium of claim 14 , wherein at least one micro-feature value associated with the micro-feature vector includes a verticality of a foreground patch representing a portion of the video scene.
18. The non-transitory processor-readable storage medium of claim 14 , further comprising determining the kinematics of the object based on at least one pixel-level characteristic of a video frame associated with the video scene.
19. The non-transitory processor-readable storage medium of claim 14 , wherein the instructions further include instructions to evaluate a rank for each cluster from the plurality of clusters relative to the micro-feature vector.
20. The non-transitory processor-readable storage medium of claim 14 , further storing processor-executable instructions to reinforce a machine learning pattern as events occur and re-occur within the video scene.