Granted Patent B2

US 12,646,096 · App. 17/930,631 · Granted Jun 2, 2026

Inventors: Michael Yamartino (San Francisco, CA); Chao Wang (San Francisco, CA); Sridatta Kaustubh Thatipamala (San Francisco, CA); Yuan Wei (San Francisco, CA)

Assignee: Pinterest, Inc.

G06Q30/0625G06Q30/0253G06Q30/0273G06Q30/0277G06Q30/0603G06Q30/0641

Abstract

Claims (90)

1 . A computer-implemented method, comprising:

receiving an image, the image including respective representations of one or more image items;

segmenting the image into a plurality of segments, the segmenting including identifying regions of the image corresponding to the one or more image items;

processing the segments of the plurality of segments containing corresponding image items, comprising:

for the one or more image items:

processing the segment corresponding to the image item to generate a first feature vector of the item represented in the segment;

comparing the first feature vector to one or more characteristics of of a plurality of item categories to determine an item category for the image item represented in the segment;

determining, for one or more items associated with the item category, a probability score indicative of a similarity between the first feature vector and a respective feature vector of the one or more items;

selecting, based at least in part on the probability scores of the one or more items, a first item of the plurality of items as the item represented in the segment of the image; and

determining a plurality of potential providers of the first item; and

presenting the image to a user, the presenting comprising:

associating a respective one of one or more controls with each of the one or more items; and

presenting the one or more controls on the image, the presenting comprising placing a control associated with a particular item at a location proximate to the corresponding segment of the image containing the particular item, wherein a first control is selectable by the user to initiate an interaction by the user with the corresponding first item from a first provider from the plurality of potential providers;

receiving a user input interacting with the first control; and

in response to the user input, generating a display presenting an additional view of the first item and a collection of item variant information associated with the item.

2 . The computer-implemented method of claim 1 , further comprising: associating the image with each of the plurality of potential providers in a mapping data store.

3 . The computer-implemented method of claim 1 , wherein the plurality of items are included in catalogs associated with the plurality of potential providers.

4 . The computer-implemented method of claim 1 , further comprising:

obtaining, from the user, an interaction with the control; and

initiating a purchase of the first item from the first provider of the plurality of potential providers.

5 . The computer-implemented method of claim 1 ,

wherein the item variant information identifies at least one of a shipping cost for the first item, a shipping time for the first item, a shipping method for the first item, a price for the first item, a size of the first item, or a color of the first item.

6 . A computing system, comprising:

one or more processors; and

a memory storing program instructions that, when executed by the one or more processors, cause the one or more processors to at least:

receive an image, the image including respective representations of one or more image items;

segment the image into a plurality of segments, the segmenting including identifying regions of the image corresponding to the one or more image items;

process the segments of the plurality of segments containing corresponding image items, comprising:

for the one or more image items:

processing the segment corresponding to the image item to generate a first feature vector of the item represented in the segment;

compare the first feature vector to one or more characteristics of a plurality of item categories to determine an item category for the image item represented in the segment;

determine, for one or more items associated with the item category, a probability score indicative of a similarity between the first feature vector and a respective feature vector of the one or more items;

selecting, based at least in part on the probability scores of the one or more items, a first item of the plurality of items as the item represented in the segment of the image; and

determine a plurality of potential providers of the first item; and

present the image to a user, the presenting comprising:

associating a respective one of one or more controls with each of the one or more items; and

presenting the one or more controls on the image, the presenting comprising placing a control associated with a particular item at a location proximate to the corresponding segment of the image containing the particular item, wherein a first control is selectable by the user to initiate an interaction with the corresponding first item from a first provider from the plurality of potential providers;

receiving a user input interacting with the first control; and

in response to the user input, generating a display presenting an additional view of the first item and a collection of item variant information associated with the item.

7 . The computing system of claim 6 , wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to at least:

determine image information associated with a first image item of the one or more image items represented in the image, wherein the image information includes a uniform resource locator (URL);

use the URL to identify one or more catalogs of items, wherein the catalogs of items comprise the plurality of items associated with the first plurality of potential providers;

generate a second feature vector of a second image item represented in the image;

determine second image information associated with the second image item represented in the image, wherein the second image information includes a second URL;

compare at least one of the second feature vector or the second image information to the plurality of items included in the one or more catalogs of items;

determine, based at least in part on the comparison of at least one of the second feature vector or the second image information to the plurality of items included in the one or more catalogs associated with the plurality of potential providers, that the second item corresponds to a third item from the plurality of items;

determine, from the plurality of potential providers and based at least in part on the third item and the one or more catalogs, a second plurality of potential providers of the third item; and

present, in association with a representation of the third item, a second control, wherein the second control provides for user interaction with the third item from a second provider from the second plurality of potential providers.

8 . The computing system of claim 6 , wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to at least:

periodically receive updates to the one or more catalogs.

9 . The computing system of claim 6 , wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to at least:

associate the image with each of a second plurality of providers in a mapping data store.

10 . The computing system of claim 6 , wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to at least:

request confirmation from the first provider that the image includes a representation of the first item; and

obtain confirmation from the first provider that the image includes the first item.

11 . The computing system of claim 6 , wherein

the item variant information identifies at least one of a shipping cost for the first item, a shipping time for the first item, a shipping method for the first item, a price for the first item, a size of the first item, or a color of the first item.

12 . The computing system of claim 11 , wherein the item variant information is obtained from a first catalog associated with the first provider.

13 . A computer-implemented method, comprising:

determining image information associated with an item represented in a user provided image, the image information including a source location identifier;

comparing the image information to a plurality of item information associated with a plurality of items included in a plurality of catalogs associated with a first plurality of providers, wherein each of the plurality of item information includes a uniform resource locator (URL);

determining that the image information matches the item information for a first item;

segmenting the image into a plurality of segments, the segmenting including identifying a region of the image corresponding to the item;

processing the segment identifying the region of the image corresponding to the item, comprising:

processing the segment to generate a first feature vector of the item represented in the segment;

comparing the first feature vector to a second feature vector for the first item;

determining, based at least in part on the comparison the item corresponds to a first item from the plurality of items;

determining, from the first plurality of providers and based at least in part on the first item and the plurality of catalogs, a second plurality of providers that provide the first item;

associating the user-provided image with the second plurality of providers in a mapping database;

obtaining, from a user, a selection of the user-provided image;

presenting the image to the user, the presenting comprising:

associating a control with the first item; and

presenting the control on the image, the presenting comprising placing the control associated with a particular item at a location proximate to the first item within the image, wherein the control is selectable by the user to initiate an interaction with the first item from a first provider from the second plurality of providers

receiving a user input interacting with the control; and

in response to the user input, generating a display presenting an additional view of the first item and a collection of item variant information associated with the item.

14 . The computer-implemented method of claim 13 , further comprising:

determining second image information associated with a second item represented in the user-provided image;

comparing the second image information to the plurality of item information associated with the plurality of items included in the plurality of catalogs associated with the first plurality of providers;

determining, based at least in part on the comparison of the second image information to the plurality of item information, that the second item corresponds to a third item from the plurality of items;

determining, from the first plurality of providers and based at least in part on the third item and the plurality of catalogs, a third plurality of providers that sell the third item;

associating the user-provided image with the third plurality of providers in the mapping database;

obtaining, from the user, a second selection of the second item represented in the user provided image; and

presenting, in association with a representation of the second item or the third item, a second control, wherein the second control enables purchase of the third item from a second provider from the third plurality of providers.

15 . The computer-implemented method of claim 14 , further comprising:

obtaining, from the user, an interaction with the second control; and initiating a purchase of the third item from the second provider.

16 . The computer-implemented method of claim 13 , wherein comparing the first feature vector to a second feature vector for the first item:

generating a probability score for the first item indicative of a similarity between the item represented in the user-provided image and the first item; and

determining that the probability score exceeds a threshold value.

17 . The computer-implemented method of claim 13 , further comprising:

determining, based at least in part on at least one of the first feature vector or the image information, an item category associated with the item represented in the user-provided image.

Assignments (2)

Continuity (4)

References Cited (90)