IP Library › Granted Patent US 11,727,054
Granted Patent B2
US 11,727,054 · App. 16/406,787 · Granted Aug 15, 2023

Method and apparatus for image recognition services

Inventors: Roopnath Grandhi (Santa Clara, CA); Raghav Gupta (Sunnyvale, CA); Neelakantan Sundaresan (Mountain View, CA); Denis Golovnya (Allston, MA); Jeffrey Olson (Medford, MA)
Assignee: eBay Inc.
G06F16/5838G06F16/434G06F16/583G06Q30/06
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,727,054
App. No.
16/406,787
Filed
May 8, 2019
Granted
Aug 15, 2023
Kind
B2
Art Unit
2164
USPC
707/705
Abstract

A system to provide image processing services responsive to requests including image data includes a system layer that forwards a request to an image application processing interface. Image processing provides an image comparison, barcode recognition, and optical character recognition. The image processing compares the image data to products in a database in order to identify a matching product. The system layer receives the matching information and forwards to a user.

Claims (72)

1. A method, comprising:

receiving, from a camera-enabled device, an image of a product, the image being taken by the camera-enabled device;

comparing, using one or more hardware processors, the received image to a plurality of stored images of a plurality of additional products;

identifying one or more similar characteristics, using the one or more hardware processors, between the received image and one or more stored images of the plurality of stored images of one or more additional products included in one or more categories of products based on the comparing; and

in response to the receiving of the image and the identifying of the one or more similar characteristics, causing enhancement of a user interface associated with the camera-enabled device, the causing of the enhancement including:

determining the one or more categories that include the one or more additional products,

generating one or more selectable identifiers of the one or more categories of products based on the image of the product and the one or more similar characteristics, each selectable identifier for selection of a particular category of products via the user interface,

generating one or more input features associated with each one or more selectable identifiers of the one or more categories of products, each input feature for receiving data associated with the product depicted in the received image,

presenting, in the user interface, the one or more selectable identifiers of the one or more categories of products and the one or more input features associated with each one or more selectable identifiers of the one or more categories of products,

presenting the one or more stored images of the one or more additional products together with the one or more selectable identifiers of the one or more categories of products, the presenting of the one or more stored images including displaying descriptions of the one or more additional products corresponding to the one or more stored images, and

receiving a request to associate the received image of the product with the one or more categories of products based on the presenting of the one or more stored images of the one or more additional products.

2. The method of claim 1 , further comprising:

receiving a selection of a selectable identifier of the one or more selectable identifiers of the one or more categories of products; and

storing the image of the product in a category associated with the selectable identifier.

3. The method of claim 1 , further comprising:

receiving input via the one or more input features associated with each one or more selectable identifiers of the one or more categories of products; and

storing the received input in association with the image of the product in a category associated with at least one of the one or more selectable identifiers.

4. The method of claim 3 , wherein the received input is in text format, the method further comprising:

updating a data dictionary to include the received input as associated with the category, the data dictionary being used in searches of products.

5. The method of claim 1 , wherein the identifying one or more similar characteristics between the received image and the one or more stored images of one or more additional products included in the one or more categories of products includes:

determining a similarity distance between the received image and the one or more stored images, the similarity distance identifying a number of matching characteristics of the received image and the one or more stored images.

6. The method of claim 1 , wherein:

the received image includes a barcode of the product; and

the identifying one or more similar characteristics between the received image and the one or more stored images of the one or more additional products included in the one or more categories of products includes matching the barcode of the product to a particular product of the one or more additional products included in the one or more categories of products.

7. The method of claim 1 , further comprising:

receiving a request to generate a listing of the product associated with the received image and a category of the one or more categories of products; and

generating the listing of the product based on the received request.

8. A system comprising:

one or more hardware processors; and

a non-transitory computer-readable medium storing instructions that, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations comprising:

receiving, from a camera-enabled device, an image of a product, the image being taken by the camera-enabled device;

comparing, using the one or more hardware processors, the received image to a plurality of stored images of a plurality of additional products;

identifying one or more similar characteristics, using the one or more hardware processors, between the received image and one or more stored images of the plurality of stored images of one or more additional products included in one or more categories of products based on the comparing; and

in response to the receiving of the image and the identifying of the one or more similar characteristics, causing enhancement of a user interface associated with the camera-enabled device, the causing of the enhancement including:

determining the one or more categories that include the one or more additional products,

generating one or more selectable identifiers of the one or more categories of products based on the image of the product and the one or more similar characteristics, each selectable identifier for selection of a particular category of products via the user interface,

generating one or more input features associated with each one or more selectable identifiers of the one or more categories of products, each input feature for receiving data associated with the product depicted in the received image,

presenting, in the user interface, the one or more selectable identifiers of the one or more categories of products and the one or more input features associated with each one or more selectable identifiers of the one or more categories of products,

presenting the one or more stored images of the one or more additional products together with the one or more selectable identifiers of the one or more categories of products, the presenting of the one or more stored images including displaying descriptions of the one or more additional products corresponding to the one or more stored images, and

receiving a request to associate the received image of the product with the one or more categories of products based on the presenting of the one or more stored images of the one or more additional products.

9. The system of claim 8 , wherein the operations further comprise:

receiving a selection of a selectable identifier of the one or more selectable identifiers of the one or more categories of products; and

storing the image of the product in a category associated with the selectable identifier.

10. The system of claim 8 , wherein the operations further comprise:

receiving input via the one or more input features associated with each one or more selectable identifiers of the one or more categories of products; and

storing the received input in association with the image of the product in a category associated with at least one of the one or more selectable identifiers.

11. The system of claim 10 , wherein the received input is in text format, and wherein the operations further comprise:

updating a data dictionary to include the received input as associated with the category, the data dictionary being used in searches of products.

12. The system of claim 8 , wherein the identifying one or more similar characteristics between the received image and the one or more stored images of one or more additional products included in the one or more categories of products includes:

determining a similarity distance between the received image and the one or more stored images, the similarity distance identifying a number of matching characteristics of the received image and the one or more stored images.

13. The system of claim 8 , wherein:

the received image includes a barcode of the product; and

the identifying one or more similar characteristics between the received image and the one or more stored images of the one or more additional products included in the one or more categories of products includes matching the barcode of the product to a particular product of the one or more additional products included in the one or more categories of products.

14. A non-transitory computer-readable medium storing instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to perform operations comprising:

receiving, from a camera-enabled device, an image of a product, the image being taken by the camera-enabled device;

comparing, using the one or more hardware processors, the received image to a plurality of stored images of a plurality of additional products;

identifying one or more similar characteristics, using the one or more hardware processors, between the received image and one or more stored images of the plurality of stored images of one or more additional products included in one or more categories of products based at least in part on the comparing; and

in response to the receiving of the image and the identifying of the one or more similar characteristics, causing enhancement of a user interface associated with the camera-enabled device, the causing of the enhancement including:

determining the one or more categories that include the one or more additional products,

generating one or more selectable identifiers of the one or more categories of products based on the image of the product and the one or more similar characteristics, each selectable identifier for selection of a particular category of products via the user interface,

generating one or more input features associated with each one or more selectable identifiers of the one or more categories of products, each input feature for receiving data associated with the product depicted in the received image,

presenting, in the user interface, the one or more selectable identifiers of the one or more categories of products and the one or more input features associated with each one or more selectable identifiers of the one or more categories of products,

presenting the one or more stored images of the one or more additional products together with the one or more selectable identifiers of the one or more categories of products, the presenting of the one or more stored images including displaying descriptions of the one or more additional products corresponding to the one or more stored images, and

receiving a request to associate the received image of the product with the one or more categories of products based on the presenting of the one or more stored images of the one or more additional products.

15. The non-transitory computer-readable medium of claim 14 , wherein the operations further comprise:

receiving a selection of a selectable identifier of the one or more selectable identifiers of the one or more categories of products; and

storing the image of the product in a category associated with the selectable identifier.

16. The non-transitory computer-readable medium of claim 14 , wherein the operations further comprise:

receiving input via the one or more input features associated with each one or more selectable identifiers of the one or more categories of products; and

storing the received input in association with the image of the product in a category associated with at least one of the one or more selectable identifiers.

17. The non-transitory computer-readable medium of claim 16 , wherein the received input is in text format, and wherein the operations further comprise:

updating a vocabulary for a data dictionary to include the received input as associated with the category, the data dictionary being used in searches of products.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 8, 2019
From: GRANDHI, ROOPNATH; GUPTA, RAGHAV; SUNDARESAN, NEELAKANTAN; GOLOVNYA, DENIS; OLSON, JEFFREY
To: EBAY INC.
Reel/Frame 049118/0645 →
Continuity (3)
Continuation 12398957 · Mar 5, 2009
Provisional Application 61033940 · Mar 5, 2008
Related Publication 20190266614A1 · Aug 29, 2019
Cited By (35)
US 12,197,610 US 12,213,090 US 12,228,944 US 12,253,617 US 12,256,021 US 12,260,616 US 12,269,498 US 12,289,181 US 12,306,010 US 12,327,445 US 12,328,639 US 12,344,168 US 12,346,712 US 12,367,718 US 12,426,007 US 12,445,285 US 12,450,329 US 12,479,446 US 12,501,178 US 12,511,947 US 12,534,097 US 12,561,624 US 12,565,143 US 12,626,200 US 12,630,050 US 12,646,402 US 12,651,529 US 12,662,152 US 12,665,989 US 12,671,464 US 12,675,419 US 12,701,004 US 12,730,425 US 12,742,846 US 12,748,942