IP Library › Granted Patent US 11,157,550
Granted Patent B2
US 11,157,550 · App. 15/025,977 · Granted Oct 26, 2021

Image search based on feature values

Inventors: Yuki Watanabe (Tokyo, JP); Atsushi Hiroike (Tokyo, JP)
Assignee: HITACHI, LTD.
G06F16/5838G06F16/248G06F16/51G06F16/532G06F16/5866
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,157,550
App. No.
15/025,977
Granted
Oct 26, 2021
Kind
B2
Abstract

An image search device stores images and tag information about the images, receives an image, extracts, from the image, feature values for search queries a similar image search unit which performs a similar-image search to obtain similar images and grouping information, divides the queries into groups in accordance with the obtained grouping information, calculates the levels of importance of the groups, sorts the search results in accordance with the levels of importance of the groups, and outputs information about the search results for each group.

Claims (47)

1. An image search system, comprising:

a storage unit that stores image feature values extracted from images to be searched and a plurality of clusters of the images to be searched;

a processor coupled to a memory, the memory storing instructions that when executed by the processor, configure the processor to:

input a plurality of images as query images;

extract image feature values from each of the query images;

search the storage unit based on the feature values of the query images and return result images having feature values within a predetermined number of the feature values of the respective query images;

determine a closest cluster based on the extracted image feature values for each of the plurality of query images among the plurality of clusters of images to be searched, wherein a cluster is two or more image feature values determined to be relatively close to one another based on a respective calculated distance between the respective image feature values,

determine a mean vector of the image feature values of each cluster, and the closest cluster of each of the plurality of query images is determined based on a respective calculated distance between the mean vector of each cluster and the image feature value of the plurality of query images,

for each of the plurality of query images, determine a similarity value of each image within the closest cluster based on a distance between each feature value within the closest cluster and the feature value of the respective query image;

determine groups of query images such that each group includes query images having a same closest cluster and each group includes the result images of the query images within the respective group;

determine an importance value of each group based on a quantity of query images in each group; and

display as a search result, the groups ordered in descending order based on the respective determined importance values, wherein each group displayed as the search result includes and displays each of the similarity values of the images of the respective group ordered in descending order based on the similarity values;

wherein the storage unit stores respective objects which are extracted from a part of each of the images to be searched, the respective objects are objects to be searched,

wherein the processor is configured to:

extract a part of the query image as a query object and extracts the image feature values from the query object,

determine a first tag for one or more of the query images based on a second tag added to a plurality of the returned result images for each of the groups, and

add the determined first tag to the one or more query images and use the newly tagged one or more query images in a subsequent search.

2. An image search method, comprising:

storing image feature values extracted from images to be searched and a plurality of clusters of the images to be searched;

inputting a plurality of images as query images;

extracting image feature values from the query images;

searching the stored image feature values to return result images having feature values within a predetermined number of the feature values of the respective query images;

determining a closest cluster based on the extracted image feature values for each of the plurality of query images among the plurality of clusters of images to be searched, wherein a cluster is two or more image feature values determined to be relatively close to one another based on a respective calculated distance between the respective image feature values,

determining a mean vector of the image feature values of each cluster, and the closest cluster of each of the plurality of query images is determined based on a respective calculated distance between the mean vector of each cluster and the image feature value of the plurality of query images,

for each of the plurality of query images, determining a similarity value of each image within the closest cluster based on a distance between each feature value within the closest cluster and the feature value of the respective query image;

determining groups of query images such that each group includes query images having a same closest cluster and each group includes the result images of the query images within the respective group;

determining an importance value of each group based on a quantity of query images in each group; and

displaying, as a search result, the groups ordered in descending order based on the respective determined importance values, wherein each group displayed as the search result includes and displays each of the similarity values of the images of the respective group ordered in descending order based on the similarity values;

determining a first tag for one or more of the query images based on a second tag added to a plurality of the returned result images for each of the groups;

adding the determined first tag to the one or more query images and use the newly tagged one or more query images in a subsequent search; and

storing respective objects which are extracted from a part of each of the images to be searched, the respective objects are objects to be searched,

wherein a part of the query image is extracted as a query object and the image feature values are extracted from the query object.

3. A non-transitory information recording medium in which there is recorded a program for instructing a computer to execute:

storing image feature values extracted from images to be searched and a plurality of clusters of the images to be searched;

inputting a plurality of images as query images;

extracting image feature values from the query images;

searching the stored image feature values to return result images having feature values within a predetermined number of the feature values of the respective query images;

determining a closest cluster based on the extracted image feature values for each of the plurality of query images among the plurality of clusters of images to be searched, wherein a cluster is two or more image feature values determined to be relatively close to one another based on a respective calculated distance between the respective image feature values,

determining a mean vector of the image feature values of each cluster, and the closest cluster of each of the plurality of query images is determined based on a respective calculated distance between the mean vector of each cluster and the image feature value of the plurality of query images,

for each of the plurality of query images, determining a similarity value of each image within the closest cluster based on a distance between each feature value within the closest cluster and the feature value of the respective query image;

determining groups of query images such that each group includes query images having a same closest cluster and each group includes the result images of the query images within the respective group;

determining an importance value of each group based on a quantity of query images in each group; and

displaying as a search result, the groups ordered in descending order based on the respective determined importance values, wherein each group displayed as the search result includes and displays each of the similarity values of the images of the respective group ordered in descending order based on the similarity values;

determining a first tag for one or more of the query images based on a second tag added to a plurality of the returned result images for each of the groups;

adding the determined first tag to the one or more query images and use the newly tagged one or more query images in a subsequent search; and

storing respective objects which are extracted from a part of each of the images to be searched, the respective objects are objects to be searched,

wherein a part of the query image is extracted as a query object and image feature values are extracted from the query object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 30, 2016
From: WATANABE, YUKI; HIROIKE, ATSUSHI
To: HITACHI, LTD.
Reel/Frame 038136/0143 →
Continuity (1)
Related Publication 20160217158A1 · Jul 28, 2016
Cited By (1)
US 12,461,961