IP Library Granted Patent US 10,289,931
Granted Patent B2
US 10,289,931 · App. 15/112,784 · Granted May 14, 2019

Method and system for searching images

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,289,931
App. No.
15/112,784
Granted
May 14, 2019
Kind
B2
Abstract

There is disclosed a method of generating an index of images, the index of images for enabling comparison of the image against other images, the method executable at a server. The method comprises: determining at least one key for the index, the at least one key including at least a portion of a visual features composite parameter associated with an image to be indexed, the visual features composite parameter having been determined by executing steps of; identifying a first local region of the image and a second local region of the image.

Claims (37)

1. A method of processing an image to enable indexing and comparison of the image against other images, the method executable at a server, the method comprising:

identifying a first local region of the image and a second local region of the image;

determining a first visual word associated with the first local region of the image and a second visual word associated with the second local region of the image, the first visual word and the second visual word being indicative of local descriptors of the first local region and of the second local region, a given local descriptor being indicative of visual features of a respective local region;

determining a visual features composite parameter associated with the image, the visual features composite parameter comprising the first visual word and the second visual word, the visual features composite parameter further comprising relationship information characterizing a relationship between the first local region and the second local region, the relationship information comprising a geometric co-location information of the first local region of the image and the second local region of the image, the geometric co-location information comprising a location of one key point associated with the first visual word relative to a location of one other key point associated with the second visual word;

determining at least one key for an index of images, the at least one key including at least a portion of the visual features composite parameter associated with the image to be indexed; and storing an indication of the image matching the visual features composite parameter in a posting list associated with the at least one key.

2. The method of claim 1 , wherein said geometric co-location information further comprises an angle associated with one key point relative to the first visual word relative to a location of one other key point associated with the second visual word.

3. The method of claim 1 , wherein said relationship information further comprises visual relationship information associated with at least one of: (i) between the first local region and the second local region, (ii) other regions of the image, and (iii) image as a whole.

4. The method of claim 3 , wherein said visual relationship information comprises at least one of respective scales relationship, respective color relationship and respective contrast relationship.

5. The method of claim 1 , wherein said relationship information further comprises at least one of; (i) visual relationship information between the first local region and the second local region, (ii) spatial information associated with other regions of the image, and (iii) visual information associated with other regions of the image.

6. The method of claim 1 , wherein: said identifying is executed on a pre-defined algorithm, and said pre-defined algorithm includes selecting the first region and the second region such that they are at least one of: (i) spaced apart within the image by a maximum possible distance, and (ii) visually as different as possible.

7. The method of claim 1 , wherein the portion comprises the first visual word and the second visual word.

8. The method of claim 1 , wherein the portion comprises the first visual word and the second visual word and at least one of: (i) a quantized spatial relationship parameter between the first region and the second region, and (ii) a quantized visual relationship parameter between the first region and the second region.

9. The method of claim 1 , wherein the portion comprises the entire visual features composite parameter, each portion of the visual features composite parameter being quantized.

10. The method of claim 1 , further comprising storing remainder of the visual features composite parameter in the posting list.

11. A method of locating a match to an image from a repository of images, the repository of images containing a plurality of images, the method executable at a server, the method comprising:

receiving an indication of the image;

determining a search image visual features composite parameter associated with the image, the visual features composite parameter further comprising relationship information characterizing a relationship between a first local region associated with a first visual word and a second local region of the image associated with a second visual word, the first visual word and the second visual word being indicative of local descriptors of the first local region and of the second local region, a given local descriptor being indicative of visual features of a respective local region, the relationship information comprising a geometric co-location information of the first local region of the image and the second local region of the image, the geometric co-location information comprising a location of one key point associated with the first visual word relative to a location of one other key point associated with the second visual word;

conducting a multi-stage search including the steps of:

selecting candidate images from the plurality of images, selecting being executed by accessing an index, the index correlating a given one of visual features composite parameters of the images contained within the plurality of images and the matching ones from the images contained within the plurality of images;

validating candidate images vis-a-vis the search image using respective local descriptors.

12. The method of claim 11 , wherein said geometric co-location information further comprises an angle associated with one key point relative to the first visual word relative to a location of one other key point associated with the second visual word.

13. The method of claim 11 , wherein said relationship information further comprises visual relationship information associated with at least one of (i) between the first Page 6 of 17 local region and the second local region, (ii) other regions of the image; (iii) image as a whole.

14. The method of claim 13 , wherein said visual relationship information comprises at least one of respective scales relationship, respective color relationship and respective contrast relationship.

15. The method of claim 11 , wherein said relationship information further comprises at least one of (i) visual relationship information between the first local region and the second local region, (ii) spatial information associated with other regions of the image, (iii) visual information associated with other regions of the image.

16. The method of claim 11 , wherein a respective one of the visual features composite parameter associated with a respective one of the images comprises a first visual word associated with a first region of the respective image, a second visual word associated with a second region of the respective image.

17. The method of claim 16 , wherein the respective one of the visual features composite parameter further comprises relationship information between the first local region and the second local region, the relationship comprising a geometric co-location information of the first local region of the image and the second local region of the image, the geometric co-location information comprising a location of one key point associated with the first visual word and a location of one other key point associated with the second visual word.

18. The method of claim 17 , wherein said geometric co-location information further comprises an angle associated with one key point relative to the first visual word relative to a location of one other key point associated with the second visual word.

19. The method of claim 17 , wherein said relationship information comprises visual relationship information between the first local region and the second local region.

20. The method of claim 19 , wherein said visual relationship information comprises at least one of respective scales relationship, respective color relationship and respective contrast relationship.

21. The method of claim 11 , further comprising prior to said receiving, populating the index using the visual features composite parameter of the images.

22. A server comprising non-transient computer-readable medium storing computer-executable instructions, which instructions when executed are configured to render the server operable to execute a method of processing an image to enable indexing and comparison of the image against other images, the method comprising: identifying a first local region of the image and a second local region of the image; determining a first visual word associated with the first local region of the image and a second visual word associated with the second local region of the image, the first visual word and the second visual word being indicative of local descriptors of the first local region and of the second local region, a given local descriptor being indicative of visual features of a respective local region; determining a visual features composite parameter associated with the image, the visual features composite parameter comprising the first visual word and the second visual word, the visual features composite parameter further comprising relationship information characterizing a relationship between the first local region and the second local region, the relationship information comprising a geometric co-location information of the first local region of the image and the second local region of the image, the geometric co-location information comprising a location of one key point associated with the first visual word Page 8 of 17 relative to a location of one other key point associated with the second visual word; determining at least one key for an index of images, the at least one key including at least a portion of the visual features composite parameter associated with the image to be indexed; and storing an indication of the image matching the visual features composite parameter in a posting list associated with the at least one key.

23. A server comprising non-transient computer-readable medium storing computer-executable instructions, which instructions when executed are configured to render the server operable to execute a method of locating a match to an image from a repository of images, the repository of images containing a plurality of images, the method comprising:

receiving an indication of the image;

determining a search image visual features composite parameter associated with the image, the visual features composite parameter further comprising relationship information characterizing a relationship between a first local region associated with a first visual word and a second local region of the image associated with a second visual word, the first visual word and the second visual word being indicative of local descriptors of the first local region and of the second local region, a given local descriptor being indicative of visual features of a respective local region, the relationship information comprising a geometric co-location information of the first local region of the image and the second local region of the image, the geometric co-location information comprising a location of one key point associated with the first visual word relative to a location of one other key point associated with the second visual word;

conducting a multi-stage search including the steps of:

selecting candidate images from the plurality of images, selecting being executed by accessing an index, the index correlating a given one of visual features composite parameters of the images contained within the plurality of images and the matching ones from the images contained within the plurality of images;

validating candidate images vis-a-vis the search image using respective local descriptors.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2024
From: DIRECT CURSUS TECHNOLOGY L.L.C
To: Y.E. HUB ARMENIA LLC
Reel/Frame 068524/0184 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2023
From: YANDEX EUROPE AG
To: DIRECT CURSUS TECHNOLOGY L.L.C
Reel/Frame 064925/0808 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2016
From: OSIPOV, ALEXANDER ALEXANDROVICH; BAYGAROVA, NATALIYA STEPANOVNA; LADAEV, DMITRY VALERIEVICH
To: YANDEX LLC
Reel/Frame 039198/0103 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2016
From: YANDEX LLC
To: YANDEX EUROPE AG
Reel/Frame 039198/0164 →