IP Library › Granted Patent US 11,907,987
Granted Patent B2
US 11,907,987 · App. 17/550,245 · Granted Feb 20, 2024

Determining visually similar products

Inventors: Estelle Afshar (Atlanta, GA); Matthew Hagen (Atlanta, GA); Huiming Qu (Atlanta, GA)
Assignee: Home Depot Product Authority, LLC
G06Q30/0623G06F18/22G06N3/045G06N3/08G06V10/761G06V10/806
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,907,987
App. No.
17/550,245
Granted
Feb 20, 2024
Kind
B2
Abstract

A computer-implemented method for determining image similarity includes determining, by a first neural network, a first feature value associated with a first characteristic of a first product based on an image of the first product. The method also includes determining, by a second neural network, a second feature value associated with a second characteristic of the first product based on the image of the first product. The method further involves calculating a first vector space distance between the first feature value and a third feature value associated with the first characteristic of a second product, and calculating a second vector space distance between the second feature value and a fourth feature value associated with the second characteristic of the second product. Additionally, the method includes determining a similarity value based on the first vector space distance and the second vector space distance.

Claims (52)

1. A computer-implemented method for determining image similarity, the method comprising:

determining, by a first neural network, a first feature value associated with a first characteristic of a first image;

determining, by a second neural network, a second feature value associated with a second characteristic of the first image;

calculating a first distance between the first feature value and a third feature value associated with the first characteristic of a second image;

calculating a second distance between the second feature value and a fourth feature value associated with the second characteristic of the second image; and

based on a comparison of the first distance to the second distance, displaying, on a user interface, a representation of the first image in association with the second image.

2. The method of claim 1 , wherein the representation of the first image is displayed in a first location within a list of images, the method further comprising:

receiving a feature weighting value associated with the first characteristic;

determining a first weighted distance based at least in part on the first distance and the feature weighting value;

calculating a weighted similarity value based on the first weighted distance and the second distance; and

determining a second location within the list of images at which to display the first image, based at least in part on the weighted similarity value.

3. The method of claim 1 , wherein the first neural network is independent from the second neural network.

4. The method of claim 1 , wherein the first image comprises a first product, and the second image comprises a second product.

5. The method of claim 1 , wherein the second characteristic is substantially independent from the first characteristic.

6. The method of claim 1 , wherein calculating the first distance comprises at least one of:

calculating, as the first distance, a cosine similarity value between the first feature value and the third feature value;

calculating, as the first distance, a Euclidean distance between the first feature value and the third feature value; or

calculating, as the first distance, a Chebyshev distance between the first feature value and the third feature value.

7. The method of claim 1 , wherein the first neural network is configured to extract one or more feature values representative of the first characteristic from input images.

8. The method of claim 1 , wherein the second neural network is configured to extract one or more feature values representative of the second characteristic from input images.

9. The method of claim 1 , further comprising:

determining, by a third neural network, a fifth feature value associated with a third characteristic of the first image.

10. The method of claim 1 , wherein the first characteristic includes color information determinable based on an input image.

11. The method of claim 1 , wherein the first characteristic includes shape information determinable based on an input image.

12. The method of claim 1 , wherein the first characteristic includes pattern information determinable based on an input image.

13. The method of claim 1 , wherein the first characteristic includes style information determinable based on an input image.

14. The method of claim 1 , wherein the comparison comprises determination of a first similarity value, the method further comprising:

determining, by the first neural network, a fifth feature value associated with the first characteristic of a third image;

determining, by the second neural network, a sixth feature value associated with the second characteristic of the third image;

determining a third distance between the third feature value associated with the second image and the fifth feature value associated with the third image;

determining a fourth distance between the fourth feature value associated with the second image and the sixth feature value associated with the third image;

determining a second similarity value based on the third distance and the fourth distance; and

based on the second similarity value being greater than the first similarity value, displaying, on the user interface, a representation of images that are visually similar to the second image in descending order, in which the third image precedes the first image.

15. A computer-implemented method for determining image similarity, the method comprising:

retrieving a set of weighting values based on a product category for a first product;

determining, by a first feature extractor, a first feature value associated with a first characteristic of an image of the first product;

determining, by a second feature extractor, a second feature value associated with a second characteristic of the first product image;

determining a first weighted distance based on the first feature value, a third feature value associated with the first characteristic of a second product image, and a first weighting value of the set of weighting values;

determining a second weighted distance based on the second feature value, a fourth feature value associated with the second characteristic of the second product image, and a second weighting value of the set of weighting values; and

in response to a comparison of the first weighted distance and the second weighted distance, displaying, on a user interface, a representation of the second product image in association with the first product image.

16. The method of claim 15 , wherein the comparison comprises:

determining a similarity value based on the first weighted distance and the second weighted distance; and

comparing the similarity value to a threshold value.

17. The method of claim 15 , wherein the representation of the first product image is displayed in a first location within a list of products, the method further comprising:

receiving a user-specified feature weighting value associated with the first characteristic based on user input;

determining a third weighted distance based on the first feature value, the user-specified feature weighting value, and the third feature value; and

based on a comparison of the third weighted distance and the second weighted distance, determining a second location within the list of products at which to display the first product image.

18. The method of claim 15 , wherein determining the first weighted distance comprises adjusting a first distance based on the first weighting value, the first distance calculated as at least one of:

a cosine similarity value between the first feature value and the third feature value;

a Euclidean distance between the first feature value and the third feature value; or

a Chebyshev distance between the first feature value and the third feature value.

19. The method of claim 15 , wherein the first feature extractor is a convolutional neural network.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2021
From: AFSHAR, ESTELLE; HAGEN, MATTHEW; QU, HUIMING
To: HOME DEPOT PRODUCT AUTHORITY, LLC
Reel/Frame 058384/0305 →
Continuity (2)
Continuation 16749629 · Jan 22, 2020
Related Publication 20220253643A1 · Aug 11, 2022
Cited By (1)
US 12,511,871