Classification of subjects within a digital image
Described herein is a system and techniques for classification of subjects within image information. In some embodiments, a set of subjects may be identified within image data obtained at two different points in time. For each of the subjects in the set of subjects, facial landmark relationships may be assessed at the two different points in time to determine a difference in facial expression. That difference may be compared to a threshold value. Additionally, contours of each of the subjects in the set of subjects may be assessed at the two different points in time to determine a difference in body position. That difference may be compared to a different threshold value. Each of the subjects in the set of subjects may then be classified based on the comparison between the differences and the threshold values.
1. A computer-implemented method comprising:
accessing image data from at least a first point in time and a second point in time, the image data comprising information associated with a set of subjects;
determining, for each subject in the set of subjects, a respective objective difference value between the first point in time and the second point in time based on:
(a) a first facial expression of the subject at the first point in time and a second facial expression of the subject at the second point in time, and
(b) a first outline contour of the subject at the first point in time and a second outline contour of the subject at the second point in time;
comparing, to an objective difference threshold, a first objective difference value for a first subject in the set of subjects and a second objective difference value for a second subject in the set of subjects; and
determining, based on the first objective difference value exceeding the objective difference threshold and the objective difference threshold exceeding the second objective difference value, that the first subject is an active subject and the second subject is a passive subject.
2. The computer-implemented method of claim 1 , wherein the first objective difference value is determined by:
identifying a first set of facial landmark locations for the first subject at the first point in time;
identifying a second set of facial landmark locations for the first subject at the second point in time; and
calculating a difference in a first relationship between the facial landmark locations in the first set of facial landmark locations and a second relationship between the facial landmark locations in the second set of facial landmark locations.
3. The computer-implemented method of claim 1 , wherein the first objective difference value is determined by:
identifying a first outline contour for the first subject at the first point in time;
identifying a second outline contour for the first subject at the second point in time; and
calculating the first objective difference value as a difference between the first outline contour for the first subject and the second outline contour for the first subject.
4. The computer-implemented method of claim 3 , wherein the first objective difference value is calculated by:
performing an exclusive-or (“XOR”) operation on a first outline contour for the first subject and a second outline contour for the first subject; and
dividing a result of the XOR operation by a size of at least one of the first outline contour for the first subject or the second outline contour for the first subject.
5. The computer-implemented method of claim 1 , wherein determining the respective objective difference value between (a) the first facial expression of the subject at the first point in time and (b) the second facial expression of the subject at the second point in time comprises:
determining a particular objective difference value between (a) the first facial expression of the subject at the first point in time and (b) the second facial expression of the subject at the second point in time; and
based on determining that the particular objective difference value is not greater than an additional threshold value, determining an additional objective difference value between (a) the first outline contour of the subject at the first point in time and (b) the second outline contour of the subject at the second point in time.
6. A system comprising:
a processor; and
a memory including instructions that, if executed with the processor, cause the system to, at least:
access image data from at least a first point in time and a second point in time, the image data comprising information associated with a set of subjects;
determine, for each subject in the set of subjects, a respective objective difference value between the first point in time and the second point in time based on:
(a) a first facial expression of the subject at the first point in time and a second facial expression of the subject at the second point in time, and
(b) a first outline contour of the subject at the first point in time and a second outline contour of the subject at the second point in time;
compare, to an objective difference threshold, a first objective difference value for a first subject in the set of subjects and a second objective difference value for a second subject in the set of subjects; and
determining, based on the first objective difference value exceeding the objective difference threshold and the objective difference threshold exceeding the second objective difference value, that the first subject is an active subject and the second subject is a passive subject.
7. The system of claim 6 , wherein determining the respective objective difference value between the first facial expression of the subject at the first point in time and the second facial expression of the subject at the second point in time comprises:
determining a first number of relationships between locations of facial landmarks for the subject at the first point in time;
determining a second number of relationships between locations of facial landmarks for the subject at the second point in time; and
determining the respective objective difference value as a difference between the first number of relationships and the second number of relationships.
8. The system of claim 6 , wherein determining the first objective difference value comprises:
determining a first outline contour for the first subject at the first point in time;
determining a second outline contour for the first subject at the second point in time; and
determining the first objective difference value as a difference between the first outline contour for the first subject and the second outline contour for the first subject.
9. The system of claim 6 , wherein determining an outline contour for the first subject comprises:
generating a bounding box at a location of the first subject;
identifying pixels within the bounding box that depict the first subject; and
creating a particular outline contour for the first subject that includes the identified pixels.
10. The system of claim 9 , wherein the bounding box is generated to have a width or height proportional to an additional width or height associated with facial features associated with the first subject.
11. The system of claim 9 , wherein generating the bounding box comprises:
placing an initial bounding box; and
moving at least one side of the initial bounding box outward from a center of the initial bounding box to form the bounding box.
12. The system of claim 11 , wherein the at least one side of the initial bounding box is moved outward from the center of the initial bounding box until it encounters either an edge of the image data or another subject.
13. The system of claim 6 , wherein comparing the objective difference threshold to the first objective difference value and to the second objective difference value further comprises:
comparing the objective difference threshold to an additional objective difference value for each additional subject in the set of subjects; and
without requiring input provided by a user, determine that at least one of the additional subjects having the additional objective difference value that does not exceed the objective difference threshold is an additional passive subject.
14. A non-transitory computer-readable medium storing specific computer-executable instructions that, if executed by a processor, cause a computer system to at least:
access image data from at least a first point in time and a second point in time, the image data comprising information associated with a set of subjects;
determine, for each subject in the set of subjects, a respective objective difference value between the first point in time and the second point in time based on:
(a) a first facial expression of the subject at the first point in time and a second facial expression of the subject at the second point in time, and
(b) a first outline contour of the subject at the first point in time and a second outline contour of the subject at the second point in time;
compare, to an objective difference threshold, a first objective difference value for a first subject in the set of subjects and a second objective difference value for a second subject in the set of subjects; and
classify, based on the first objective difference value exceeding the objective difference threshold and the objective difference threshold exceeding the second objective difference value, that the first subject is an active subject and that the second subject is a passive subject.
15. The computer-readable medium of claim 14 , wherein the respective objective difference value is determined by:
identifying a first set of facial landmark locations for the subject at the first point in time;
identifying a second set of facial landmark locations for the subject at the second point in time; and
calculating a difference in a first relationship between the facial landmark locations in the first set of facial landmark locations and a second relationship between the facial landmark locations in the second set of facial landmark locations.
16. The computer-readable medium of claim 14 , wherein the first objective difference value is determined by:
identifying a first outline contour for the first subject at the first point in time;
identifying a second outline contour for the first subject at the second point in time; and
calculating the first objective difference value as a difference between the first outline contour for the first subject and the second outline contour for the first subject.
17. The computer-readable medium of claim 14 , wherein the image data from the first point in time and the second point in time are included within a live capture event.
18. The computer-readable medium of claim 14 , wherein the image data from the first point in time and the second point in time are selected from a user's catalog of images based upon the image data from the first point in time and the second point in time both including the set of subjects.
19. The computer-readable medium of claim 14 , wherein a value of the objective difference threshold is determined based on one or more characteristics of the image data.
20. The computer-readable medium of claim 19 , wherein the one or more characteristics of the image data comprises a motion of a camera at the first point in time or the second point in time.