IP Library Granted Patent US 10,977,528
Granted Patent B1
US 10,977,528 · App. 16/452,093 · Granted Apr 13, 2021

Detecting similarity between images

Inventor: Dylan Tong (Mercer Island, WA)
Assignee: Amazon Technologies, Inc.
G06K9/66G06K9/6215
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,977,528
App. No.
16/452,093
Granted
Apr 13, 2021
Kind
B1
Abstract

Techniques for determining image similar are described. For example, a computer-implemented method comprising: receiving a request to determine similarity between a first image and at least one other image; determining similarity between the first image and at least one other image based upon one or more Gram matrix-based style values and one or more vector distance calculation-based content values as determined from one or more outputs of layers of a convolutional neural network; and providing an indication of the similarity of between the first image and the at least one other image is described.

Claims (31)

1. A computer-implemented method comprising:

receiving a request to display an interface including an item that has an associated image;

requesting for, and receiving, a determination one or more similarity scores between the item and items of a same or similar category as the item, the one or more similarity scores based upon one or more Gram matrix-based style values and one or more vector distance calculation-based content values as determined from one or more outputs of layers of a convolutional neural network; and

displaying one or more images of items that are similar to the image based on the one or more similarity scores.

2. The computer-implemented method of claim 1 , wherein the convolutional neural network is a pre-trained image feature extraction model.

3. The computer-implemented method of claim 1 , wherein which layers of the convolutional neural network to use for the Gram matrix-based and vector distance calculation-based content values are configurable.

4. A computer-implemented method comprising:

receiving a request to determine similarity between a first image and at least one other image;

determining similarity between the first image and at least one other image based upon one or more Gram matrix-based style values and one or more vector distance calculation-based content values as determined from one or more outputs of layers of a convolutional neural network; and

providing an indication of the similarity between the first image and the at least one other image.

5. The computer-implemented method of claim 4 , wherein the convolutional neural network is a pre-trained image feature extraction model.

6. The computer-implemented method of claim 5 , wherein the vector distance calculation is one of L1 or L2 distance.

7. The computer-implemented method of claim 4 , wherein which layers of the convolutional neural network to use for the Gram matrix-based and vector distance calculation-based content values are configurable.

8. The computer-implemented method of claim 4 , wherein the indication is a similarity value generated by determining a difference of blended one Gram matrix-based style values and one or more vector distance calculation-based content values of the first image and blended one Gram matrix-based style values and one or more vector distance calculation-based content values of the at least one other image.

9. The computer-implemented method of claim 4 , wherein the similarity between the first image and at least one other image is determined by an indexed database query to find images having similar Gram-based style values and vector distance calculation-based content values as the first image.

10. The computer-implemented method of claim 9 , wherein the indication is one or more locations of images that have similar Gram-based style values and vector distance calculation-based content values.

11. The computer-implemented method of claim 4 , wherein the similarity between the first image and at least one other image is determined by a nearest neighbor search against an index to find images having similar Gram-based style values and vector distance calculation-based content values as the first image.

12. The computer-implemented method of claim 4 , wherein the indication is one or more locations of images that have similar Gram-based style values and L2-based content values.

13. The computer-implemented method of claim 4 , wherein the convolutional neural network has been configured using a configuration having on or more of: content extraction layers identification and respective layer weightings, style extraction layers identification and respective layer weightings, a ratio of content versus style influence, and image storage information.

14. The computer-implemented method of claim 4 , further comprising:

receiving an input to change a configuration of which layers of the convolutional neural network are to be used to determine similarity between the first image and at least one other image.

15. A system comprising:

a web application implemented by a first one or more electronic devices; and

a neural style inference service implemented by a second one or more electronic devices, the neural style inference service including instructions that upon execution cause the neural style inference service to: receive a request from the web application to determine similarity between a first image and at least one other image;

determine similarity between the first image and at least one other image based upon one or more Gram matrix-based style values and one or more vector distance calculation-based content values as determined from one or more outputs of layers of a convolutional neural network; and

provide an indication of the similarity between the first image and the at least one other image.

16. The system of claim 15 , wherein the convolutional neural network is a pre-trained image feature extraction.

17. The system of claim 15 , wherein the vector distance calculation is one of L1 or L2 distance.

18. The system of claim 15 , wherein which layers of the convolutional neural network to use for the Gram matrix-based and vector distance calculation-based content values are configurable.

19. The system of claim 15 , wherein the similarity between the first image and at least one other image is determined by a nearest neighbor search against an index to find images having similar Gram-based style values and vector distance calculation-based content values as the first image.

20. The system of claim 15 , wherein the convolutional neural network has been configured using a configuration having on or more of: content extraction layers identification and respective layer weightings, style extraction layers identification and respective layer weightings, a ratio of content versus style influence, and image storage information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 26, 2019
From: TONG, DYLAN
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 049590/0634 →
Cited By (3)
US 12,235,731 US 12,354,324 US 12,597,239