System and method for age estimation
Systems, methods, and computer-readable storage media for age estimation/classification, and more specifically to estimating/classifying the ages of people appearing within videos. Systems configured as disclosed herein can receive a video, then identify multiple facial images for each individual captured within the video. The system can create embeddings of the facial images, then cluster those images together based on distances between the corresponding embeddings. The system can also execute a matchability algorithm on those facial images, determining which of the images provides the clearest image of the individual(s), and can then estimate the age of the individual(s) using the best matchability images and/or send the best matchability image for each individual to a third party for analysis.
1. A method comprising:
receiving, at a first computer system, a video, the video having a plurality of frames;
performing, via at least one processor of the first computer system, facial detection on the plurality of frames, resulting in a plurality of facial images;
executing, via the at least one processor, a matchability algorithm on the plurality of facial images, resulting in a plurality of matchability scores, each matchability score in the plurality of matchability scores corresponding to a likelihood that a matching image will be found within the plurality of facial images;
executing, via the at least one processor, an embedding algorithm on the plurality of facial images, resulting in a plurality of embeddings, each embedding in the plurality of embeddings corresponding to an image within the plurality of facial images;
calculating, via the at least one processor, distances between the plurality of embeddings, resulting in a Euclidean distance matrix;
clustering, via the at least one processor, the plurality of embeddings using the Euclidean distance matrix, resulting in at least one cluster of embeddings;
identifying, within the plurality of facial images, a best facial image for each cluster in the at least one cluster of embeddings based on the plurality of matchability scores;
transmitting, from the first computer system to a third party computer system which is distinct from the first computer system, an age estimation request, the age estimation request comprising the best facial image of an individual and a request to estimate an age of the individual corresponding to the best facial image;
receiving, at the first computer system from the third party computer system, in response to the age estimation request, an indication that the age of the individual corresponding to the best facial image is below a predetermined threshold age; and
flagging, via the at least one processor, the video for human review based on the indication.
2. The method of claim 1 , wherein the identifying of the best facial image for each individual captured in the video further comprises:
extracting, via the at least one processor of the computer system, at least two frames from the video, resulting in a plurality of frames; and
executing, via the at least one processor, the facial detection on each frame the plurality of frames, resulting in the plurality of facial images.
3. The method of claim 1 , wherein executing of the matchability algorithm comprises generating, for each facial image in the plurality of facial images, emotion scores, wherein the emotion scores are used to compute the plurality of matchability scores.
4. The method of claim 3 , wherein the emotion scores comprise, for each facial image in the plurality of facial images, a vector of scores for emotions comprising: happiness, surprise, sadness, anger, disgust, fear, and contempt.
5. The method of claim 1 , wherein executing of the matchability algorithm comprises generating, for each facial image in the plurality of facial images, a facial angle, wherein the facial angle is used to compute the plurality of matchability scores.
6. The method of claim 1 , further comprising:
executing, via the at least one processor, an internal age classification algorithm on the at least one cluster of embeddings, resulting in an age classification for each individual appearing in the video, prior to the transmitting of the age estimation request,
wherein the transmitting of the age estimation request is modified based on the age classification of each individual appearing in the video.
7. The method of claim 6 , wherein the internal age classification algorithm uses the plurality of embeddings to estimate an age of each individual appearing in the video.
8. A system comprising:
at least one processor; and
a non-transitory computer-readable storage medium having instructions stored which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
receiving a video, the video having a plurality of frames;
performing facial detection on the plurality of frames, resulting in a plurality of facial images;
executing a matchability algorithm on the plurality of facial images, resulting in a plurality of matchability scores, each matchability score in the plurality of matchability scores corresponding to a likelihood that a matching image will be found within the plurality of facial images;
executing an embedding algorithm on the plurality of facial images, resulting in a plurality of embeddings, each embedding in the plurality of embeddings corresponding to an image within the plurality of facial images;
calculating distances between the plurality of embeddings, resulting in a Euclidean distance matrix;
clustering the plurality of embeddings using the Euclidean distance matrix, resulting in at least one cluster of embeddings;
identifying, within the plurality of facial images, a best facial image for each cluster in the at least one cluster of embeddings based on the plurality of matchability scores;
transmitting, to a third party computer system which is distinct from the system, an age estimation request, the age estimation request comprising the best facial image of an individual and a request to estimate an age of the individual corresponding to the best facial image;
receiving, from the third party computer system, in response to the age estimation request, an indication that the age of the individual corresponding to the best facial image is below a predetermined threshold age; and
flagging the video for human review based on the indication.
9. The system of claim 8 , wherein the identifying of the best facial image for each individual captured in the video further comprises:
extracting at least two frames from the video, resulting in a plurality of frames; and
executing, via the at least one processor, the facial detection on each frame the plurality of frames, resulting in the plurality of facial images.
10. The system of claim 8 , wherein executing of the matchability algorithm comprises generating, for each facial image in the plurality of facial images, emotion scores, wherein the emotion scores are used to compute the plurality of matchability scores.
11. The system of claim 10 , wherein the emotion scores comprise, for each facial image in the plurality of facial images, a vector of scores for emotions comprising: happiness, surprise, sadness, anger, disgust, fear, and contempt.
12. The system of claim 8 , wherein executing of the matchability algorithm comprises generating, for each facial image in the plurality of facial images, a facial angle, wherein the facial angle is used to compute the plurality of matchability scores.
13. The system of claim 8 , the non-transitory computer-readable storage medium having additional instructions stored which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
executing an internal age classification algorithm on the at least one cluster of embeddings, resulting in an age classification for each individual appearing in the video, prior to the transmitting of the age estimation request,
wherein the transmitting of the age estimation request is modified based on the age classification of each individual appearing in the video.
14. The system of claim 13 , wherein the internal age classification algorithm uses the plurality of embeddings to estimate an age of each individual appearing in the video.
15. A non-transitory computer-readable storage medium having instructions stored which, when executed by at least one processor, cause the at least one processor to perform operations comprising:
receiving a video, the video having a plurality of frames;
performing facial detection on the plurality of frames, resulting in a plurality of facial images;
executing a matchability algorithm on the plurality of facial images, resulting in a plurality of matchability scores, each matchability score in the plurality of matchability scores corresponding to a likelihood that a matching image will be found within the plurality of facial images;
executing an embedding algorithm on the plurality of facial images, resulting in a plurality of embeddings, each embedding in the plurality of embeddings corresponding to an image within the plurality of facial images;
calculating distances between the plurality of embeddings, resulting in a Euclidean distance matrix;
clustering the plurality of embeddings using the Euclidean distance matrix, resulting in at least one cluster of embeddings;
identifying, within the plurality of facial images, a best facial image for each cluster in the at least one cluster of embeddings based on the plurality of matchability scores; and
transmitting, to a third party computer system which is physically separated from the at least one processor, an age estimation request, the age estimation request comprising the best facial image of an individual and a request to estimate an age of the individual corresponding to the best facial image;
receiving, from the third party computer system, in response to the age estimation request, an indication that the age of the individual corresponding to the best facial image is below a predetermined threshold age; and
flagging the video for human review based on the indication.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the identifying of the best facial image for each individual captured in the video further comprises:
extracting at least two frames from the video, resulting in a plurality of frames; and
executing the facial detection on each frame the plurality of frames, resulting in the plurality of facial images.
17. The non-transitory computer-readable storage medium of claim 15 , wherein executing of the matchability algorithm comprises generating, for each facial image in the plurality of facial images, emotion scores, wherein the emotion scores are used to compute the plurality of matchability scores.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the emotion scores comprise, for each facial image in the plurality of facial images, a vector of scores for emotions comprising: happiness, surprise, sadness, anger, disgust, fear, and contempt.
19. The non-transitory computer-readable storage medium of claim 15 , wherein executing of the matchability algorithm comprises generating, for each facial image in the plurality of facial images, a facial angle, wherein the facial angle is used to compute the plurality of matchability scores.
20. The non-transitory computer-readable storage medium of claim 15 , having additional instructions stored which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
executing an internal age classification algorithm on the at least one cluster of embeddings, resulting in an age classification for each individual appearing in the video, prior to the transmitting of the age estimation request,
wherein the transmitting of the age estimation request is modified based on the age classification of each individual appearing in the video.