Image processing apparatus, image processing method, and non-transitory storage medium
To reduce a load on a computer required for processing of searching for a desired person from among a plurality of images, the present invention provides an information processing apparatus 10 including: an acquisition unit 11 that acquires a query indicating a person attribute; and a search unit 12 that refers to a reference image database in which a plurality of reference images are grouped by putting together into reference images including an identical person and clustered, for each group, by putting together those having a similar predetermined attribute, selects, for each group, a representative reference image from one or each of a plurality of clusters, and collates the selected representative reference image with the query, based on the person attribute.
1 . An image processing apparatus comprising:
at least one memory configured to store one or more instructions; and
at least one processor configured to execute the one or more instructions to:
group a plurality of reference images into one or more groups, wherein each group of the one or more groups comprises one or more first reference images from the plurality of reference images, the one or more first reference images corresponding to a same person;
cluster the one or more first reference images in each of the one or more groups into one or more clusters, wherein each cluster of the one or more clusters comprises one or more second reference images from the one or more first reference images, the one or more second reference images having a similar predetermined attribute based on (i) matching, (ii) a similarity condition, or (iii) a mutual difference with respect to a threshold value;
create a reference image database by associating reference image identification information for the plurality of reference images with group identification information for the one or more groups and cluster identification information for the one or more clusters;
acquire a query indicating a person attribute;
select, for each group of the one or more groups, one or more representative reference images by:
based on the query further indicating the predetermined attribute, selecting the one or more representative reference images from a cluster of the one or more clusters associated with the predetermined attribute; and
based on the query not indicating the predetermined attribute, selecting the one or more representative reference images from each cluster of the one or more clusters associated with each group;
collate the one or more selected representative reference images with the query, based on the person attribute by identifying, from among the one or more selected representative reference images, a target representative reference image that comprises a person having the person attribute indicated in the query; and
output, based on the target representative reference image, a search result comprising at least one of the plurality of reference images belonging to a same group and a same cluster as the target representative reference image.
2 . The image processing apparatus according to claim 1 , wherein
the person attribute indicates a person's appearance attribute or a person's pose.
3 . The image processing apparatus according to claim 2 , wherein
the person's appearance attribute includes at least one of age, gender, a feature of a wearable article, a body shape, nationality, and a hairstyle.
4 . The image processing apparatus according to claim 1 , wherein
the predetermined attribute includes at least one of a person's pose, a person's appearance attribute, a position in an image where a person is present, a photographing time zone, a photographing place, a model of a photographing camera, and a type of a key point on a person's body detected from an image.
5 . The image processing apparatus according to claim 4 , wherein
the person attribute includes a person's appearance attribute, and
the predetermined attribute includes at least one of a person's pose, a position in an image where a person is present, a photographing time zone, a photographing place, a model of a photographing camera, and a type of a key point on a person's body detected from an image.
6 . The image processing apparatus according to claim 5 , wherein
the query indicates a person's appearance attribute and a person's pose.
7 . The image processing apparatus according to claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to
acquire, as the query, a moving image including a person,
group a plurality of frame images included in the moving image into one or more frame groups, wherein each frame group comprises one or more first frame images from the plurality of frame images, the one or more first frame images corresponding to a same person, and cluster the one or more first frame images in each of the one or more frame groups into one or more frame clusters, wherein each frame cluster comprises one or more second frame images from the one or more first frame images, the one or more second frame images having a similar predetermined attribute,
select, for each frame group, one or more representative frame images from a cluster of the one or more frame clusters, and
collate the one or more selected representative frame images with the one or more selected representative reference images.
8 . The image processing apparatus according to claim 1 , wherein the plurality of reference images correspond to a moving image having a duration of at least one hour at 30 or more frames per second.
9 . The image processing apparatus according to claim 1 , wherein the target representative reference image is identified, from among the one or more selected representative reference images, based on a predetermined rule.
10 . The image processing apparatus according to claim 1 , wherein the target representative reference image is identified by randomly selecting an image from among the one or more selected representative reference images.
11 . An image processing method, performed by one or more computers, the method comprising:
grouping a plurality of reference images into one or more groups, wherein each group of the one or more groups comprises one or more first reference images from the plurality of reference images, the one or more first reference images corresponding to a same person;
clustering the one or more first reference images in each of the one or more groups into one or more clusters, wherein each cluster of the one or more clusters comprises one or more second reference images from the one or more first reference images, the one or more second reference images having a similar predetermined attribute based on (i) matching, (ii) a similarity condition, or (iii) a mutual difference with respect to a threshold value;
creating a reference image database by associating reference image identification information for the plurality of reference images with group identification information for the one or more groups and cluster identification information for the one or more clusters;
acquiring a query indicating a person attribute;
selecting, for each group of the one or more groups, one or more representative reference images by:
based on the query further indicating the predetermined attribute, selecting the one or more representative reference images from a cluster of the one or more clusters associated with the predetermined attribute; and
based on the query not indicating the predetermined attribute, selecting the one or more representative reference images from each cluster of the one or more clusters associated with each group;
collating the one or more selected representative reference images with the query, based on the person attribute by identifying, from among the one or more selected representative reference images, a target representative reference image that comprises a person having the person attribute indicated in the query; and
outputting, based on the target representative reference image, a search result comprising at least one of the plurality of reference images belonging to a same group and a same cluster as the target representative reference image.
12 . A non-transitory storage medium storing a program causing a computer to:
group a plurality of reference images into one or more groups, wherein each group of the one or more groups comprises one or more first reference images from the plurality of reference images, the one or more first reference images corresponding to a same person;
cluster the one or more first reference images in each of the one or more groups into one or more clusters, wherein each cluster of the one or more clusters comprises one or more second reference images from the one or more first reference images, the one or more second reference images having a similar predetermined attribute based on (i) matching, (ii) a similarity condition, or (iii) a mutual difference with respect to a threshold value;
create a reference image database by associating reference image identification information for the plurality of reference images with group identification information for the one or more groups and cluster identification information for the one or more clusters;
acquire a query indicating a person attribute;
select, for each group of the one or more groups, one or more representative reference images by:
based on the query further indicating the predetermined attribute, selecting the one or more representative reference images from a cluster of the one or more clusters associated with the predetermined attribute; and
based on the query not indicating the predetermined attribute, selecting the one or more representative reference images from each cluster of the one or more clusters associated with each group;
collate the one or more selected representative reference images with the query, based on the person attribute by identifying, from among the one or more selected representative reference images, a target representative reference image that comprises a person having the person attribute indicated in the query; and
output, based on the target representative reference image, a search result comprising at least one of the plurality of reference images belonging to a same group and a same cluster as the target representative reference image.