System and method for identifying a person in a video
Systems, methods, and computer readable media for identifying a person in a video are disclosed. Systems, methods, devices, and non-transitory computer readable media may include at least one processor that may be configured to generate a spatiotemporal emotion data compendium (STEM-DC) from the video and to process the STEM-DC using a deep fully adaptive graph convolutional network (FAGC) to determine a first person representation vector that represents the person in the video.
1. A system for identifying a person in a video, comprising:
a computing device configured to generate a spatiotemporal emotion data compendium (STEM-DC) from the video; and to process the STEM-DC using a deep fully adaptive graph convolutional network (FAGC) to determine a first person representation vector that represents the person in the video,
wherein the generating the STEM-DC includes generating an iterated feature vector (IFV), and wherein the generating the IFV includes iterating a series of landmark feature vectors weighted by functions of transition probabilities between basic emotional states of the person detected in subsequent frames of the video.
2. The system of claim 1 , further configured to compare the first person representation vector with a subsequent second person representation vector determined from a subsequent video and subsequent STEM-DC, to thereby identify the person as appearing in the subsequent video when the first and second person representation vectors are substantially similar.
3. The system of claim 2 , wherein the first and second person representation vectors are based on an identifiable trait of the person in the video.
4. The system of claim 3 , wherein the identifiable trait includes at least one of face, emotion, gait, body, limb, or typing style.
5. The system of claim 1 , wherein the functions of transition probabilities are represented by a transition weight sum matrix.
6. The system of claim 1 , wherein each of the basic emotional states is determined by projecting an emotion feature vector onto a series of emotion basis vectors.
7. The system of claim 1 , wherein each of the series of landmark feature vectors for a given facial image includes L landmarks characterized by P features.
8. The system of claim 7 , wherein the series of landmark feature vectors is determined by processing facial images extracted from the video using a pretrained facial landmark extraction net (FLEN), to identify the L facial landmarks each characterized by P features.
9. The system of claim 8 , wherein the facial images from the video are extracted by locating and rectifying images of a person's face located in the video.
10. The system of claim 1 , wherein the FAGC includes a feature extraction module and a data merging module that includes a plurality of convolution blocks.
11. The system of claim 1 , wherein a resolution of the basic emotional states is increased for a video having a higher frame rate.