Method and apparatus for organizing digital media based on face recognition
View Patent ↗In one aspect, the present invention is directed to a method and an apparatus for organizing digital media, particularly digital photos, using face recognition. According to a first aspect of the present invention, a computer-based method for organizing digital photos comprises: extracting objects of interest from a plurality of photographs; cropping said plurality of photographs to generate images of isolated objects of interest; applying a recognition algorithm to determine the similarity of isolated objects of interest with a reference; displaying a plurality of objects arranged as a function of the determined similarity; and receiving user input to associate said objects with a particular classification.
1. A computer-based method for organizing digital photos, comprising:
(a) detecting and cropping out faces from a plurality of digital photos in a first group to generate isolated face images;
(b) applying a face recognition algorithm to determine similarity of each of the isolated face images contained in the digital photos in the first group with a model face image set, wherein the model face image set includes at least one model face image contained in a digital photo in a second group;
(c) displaying the isolated face images on a display screen in accordance with the determined similarity;
(d) receiving a user input to select one of the isolated face images displayed on the display screen;
(e) reclassifying the digital photo containing the selected one of the isolated face images into the second group instead of the first group;
(f) modifying the model face image set by adding the selected one of the isolated face images;
(g) renewing the determined similarity by applying the face recognition algorithm to determine similarity of each of the isolated face images contained in the digital photos in the first group with the modified model face image set; and
(h) displaying the isolated face images on the display screen in accordance with the renewed determined similarity.
2. The method as defined in claim 1 , wherein in the step (c), each of the isolated face images is displayed in a view that includes an area surrounding the face.
3. The method as defined in claim 1 , further comprising:
(i) after the step (h), repeatedly executing the steps (d), (e), (f), (g) and (h).
4. The method as defined in claim 1 , wherein:
in the step (c), the isolated face images are displayed on the display screen in a first area; and
in the step (d), the user input drags the selected one of the isolated face images on the display screen from the first area into a second area in which the model face image set is to be displayed.
5. The method as defined in claim 4 , further comprising:
(j) before the step (d), displaying the model face image set on the display screen in the second area.
6. The method as defined in claim 1 , further comprising:
(k) before the step (b), receiving a user input to designate the model face image; and
(l) after the step (k) and before the step (b), preparing the model face image set including the designated model face image.
7. The method as defined in claim 6 , wherein:
the second group has been associated with a particular person; and
in the step (k), the user input selects the second group among a plurality of groups respectively associated with a plurality of persons.
8. The method as defined in claim 6 , further comprising:
(m) before the step (k), displaying the isolated face images on the display screen, wherein in the step (k), the user input selects one of the isolated face images displayed on the display screen as the model face image; and
(n) after the step (k) and before the step (b), reclassifying the digital photo containing the selected one of the isolated face images selected in the step (k) into the second group instead of the first group.
9. The method as defined in claim 8 , wherein:
in the step (m), the isolated face images are displayed on the display screen in a first area; and
in the step (k), the user input drags the selected one of the isolated face images on the display screen from the first area into a second area in which the model face image set is to be displayed.
10. The method as defined in claim 1 , wherein in the step (c), the isolated face images are displayed in order of the determined similarity.
11. The method as defined in claim 1 , wherein in the step (h), the isolated face images are displayed in order of the renewed determined similarity.
12. An apparatus for organizing digital photos, comprising:
a face detection and cropping unit configured to detect and crop out faces from a plurality of digital photos in a first group to generate isolated face images;
a face recognition unit configured to apply a face recognition algorithm to determine similarity of each of the isolated face images contained in the digital photos in the first group with a model face image set, wherein the model face image set includes at least one model face image contained in a digital photo in a second group;
a display output unit configured to output a display of the isolated face images on a display screen in accordance with the determined similarity;
a user input unit configured to receive user input; and
a media organizing control unit configured
to cause the user input unit to receive a user input to select one of the isolated face images displayed on the display screen,
to reclassify the digital photo containing the selected one of the isolated face images into the second group instead of the first group,
to modify the model face image set by adding the selected one of the isolated face images,
to renew the determined similarity by causing the face recognition unit to apply the face recognition algorithm to determine similarity of each of the isolated face images contained in the digital photos in the first group with the modified model face image set, and
to cause the display output unit to output a display of the isolated face images on the display screen in accordance with the renewed determined similarity.
13. The apparatus as defined in claim 12 , wherein the display output unit is configured to output the display of each of the isolated face images in a view that includes an area surrounding the face.
14. The apparatus as defined in claim 12 , wherein the media organizing control unit is configured to repeatedly execute, after causing the display output unit to output the display of the isolated face images on the display screen in accordance with the renewed determined similarity,
causing the user input unit to receive a user input to select one of the isolated face images displayed on the display screen,
reclassifying the digital photo containing the selected one of the isolated face images into the second group instead of the first group,
modifying the model face image set by adding the selected one of the isolated face images,
renewing the determined similarity by causing the face recognition unit to apply the face recognition algorithm to determine similarity of each of the isolated face images contained in the digital photos in the first group with the modified model face image set, and
causing the display output unit to output a display of the isolated face images on the display screen in accordance with the renewed determined similarity.
15. The apparatus as defined in claim 12 , wherein:
the display output unit is configured to output the display of the isolated face images on the display screen in a first area, and to output a display of the model face image set on the display screen in a second area; and
the media organizing control unit is configured to cause the user input unit to receive the user input dragging the selected one of the isolated face images on the display screen from the first area into the second area.
16. The apparatus as defined in claim 12 , wherein the media organizing control unit is configured to cause the user input unit to receive a user input to designate the model face image, and is configured to prepare the model face image set including the designated model face image.
17. The apparatus as defined in claim 16 , wherein:
the second group has been associated with a particular person; and
the media organizing control unit is configured to cause the user input unit to receive the user input selecting the second group among a plurality of groups respectively associated with a plurality of persons.
18. The apparatus as defined in claim 16 , wherein:
the display output unit is configured to output a display of the isolated face images on the display screen before the face recognition unit determines the similarity of each of the isolated face images; and
the media organizing control unit is configured to cause the user input unit to receive the user input selecting one of the isolated face images displayed on the display screen as the model face image, and is configured to reclassify the digital photo containing the selected one of the isolated face images selected by the user input into the second group instead of the first group.
19. The apparatus as defined in claim 18 , wherein:
the display output unit is configured to output the display of the isolated face images on the display screen in a first area, and to output a display of the model face image set on the display screen in a second area; and
the media organizing control unit is configured to cause the user input unit to receive the user input dragging the selected one of the isolated face images on the display screen from the first area into the second area.
20. The apparatus as defined in claim 12 , wherein the media organizing control unit is configured to cause the display output unit to output the display of the isolated face images on the display screen in order of the renewed determined similarity.