IP Library Granted Patent US 10,242,099
Granted Patent B1
US 10,242,099 · App. 13/448,049 · Granted Mar 26, 2019

Cascaded multi-tier visual search system

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,242,099
App. No.
13/448,049
Granted
Mar 26, 2019
Kind
B1
Abstract

A system for multi-tier visual recognition comprises a processor and a memory. The processor is configured to calculate a rank for each of a plurality of object models. The plurality of object models are used for visual recognition. Each of the plurality of object models has a size. The processor is configured to divide the plurality of object models into a plurality of sets based at least in part on the rank for each of the plurality of object models, and provide a first set of the plurality of sets to a first tier and a second set of the plurality of sets to a second tier. The memory coupled to the processor and configured to provide the processor with instructions.

Claims (44)

1. A system for search of image data, comprising:

a processor; and

a memory having stored thereon instructions that, when executed by the processor, cause the processor to:

receive the image data from a client device of a user,

obtain a plurality of object models used for visual recognition of objects in the image data, wherein each of the plurality of object models is associated with an image object capable of being identified within an image,

determine a rank for each of the plurality of object models,

identify a first set of the plurality of object models to be used by the client device and a second set of the plurality of object models to be used by a server based on the rank, and

provide the first set of the plurality of object models to the client device and the second set of the plurality of object models to the server so that the client device performs a first search for a first set of objects in the image data based on the first set of the plurality of object models and the server performs a second search for a second set of objects in the image data based on the second set of the plurality of object models.

2. The system as in claim 1 , wherein the first set of the plurality of object models are identified based, at least in part, upon storage capacity of the client device.

3. The system as in claim 2 , wherein the first set of the plurality of object models are identified based, at least in part, upon a size associated with each object model and the rank associated with each object model.

4. The system as in claim 1 , wherein the visual recognition comprises recognizing at least one of: barcode type, icons, logos, advertisements, media, brands, products, geography, and structures.

5. The system as in claim 4 , wherein:

at least one of the server and the client device are configured to provide a result for a first object identified in the image data using one of the plurality of object models; and

the result comprises one or more of: text tag, metadata, results page, results indicator, vocalized tag, visual animation, an action such as a redirection to a webpage or a video, a visual ranking of objects, or a vocalized ranking of objects.

6. A method implemented on a computing device having at least one processor, storage, and communication platform for search of image data, comprising:

receiving the image data from a client device of a user;

obtaining a plurality of object models used for visual recognition of objects in the image data, wherein each of the plurality of object models is associated with an image object capable of being identified within an image;

determining, by the processor, a rank for each of the plurality of object models;

identifying a first set of the plurality of object models to be used by the client device and a second set of the plurality of object models to be used by a server based on the rank; and

providing the first set of the plurality of object models to the client device and the second set of the plurality of object models to the server so that the client device performs a first search for a first set of objects in the image data based on the first set of the plurality of object models and the server performs a second search for a second set of objects in the image data based on the second set of the plurality of object models.

7. A computer program product for search of image data, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer executable instructions for:

receiving the image data from a client device of a user;

obtaining a plurality of object models used for visual recognition of objects in the image data, wherein each of the plurality of object models is associated with an image object capable of being identified within an image;

determining, by the processor, a rank for each of the plurality of object models;

identifying a first set of the plurality of object models to be used by the client device and a second set of the plurality of object models to be used by a server based on the rank; and

providing the first set of the plurality of object models to the client device and the second set of the plurality of object models to the server so that the client device performs a first search for a first set of objects in the image data based on the first set of the plurality of object models and the server performs a second search for a second set of objects in the image data based on the second set of the plurality of object models.

8. The method as in claim 6 , further comprising:

determining a change in the rank of at least one of the plurality of object models;

wherein the first set of the plurality of object models to be provided to the client device is identified based, at least in part, on the change in the rank of the at least one of the plurality of object models.

9. The method as in claim 8 , wherein the change in the rank of at least one of the plurality of object models is triggered by at least one of: a change of location of the client device, a change of user preference, a new search history pattern, feedback from the user, a change of friend or family information, and a local event.

10. The method of claim 6 , further comprising:

determining that there is an update to at least one of the first set and the second set; and

transmitting the update to the at least one of the first set and the second set to the client device.

11. The method as in claim 10 , wherein a timing of the update being transmitted to the client device is based on one or more thresholds.

12. The method as in claim 11 , wherein at least one of the one or more thresholds is user configurable or tailored to the user.

13. The method as in claim 6 , wherein the client device comprises a mobile device, a fixed device, a phone, a tablet, a camera, a laptop, a desktop computer, a webcam, an appliance, or any other imaging device.

14. The method as in claim 6 , wherein the relevance is determined based, at least in part, upon at least one of a location of the user, a user preference, a search history pattern, feedback from the user, friend or family information, or a local event.

15. The method as in claim 6 , wherein the ranking is performed based, at least in part, upon an ontology feature or a collaborative feature.

16. The method of claim 6 , further comprising:

determining, in response to the client device performing the first search for the first set of objects in the image data, that at least one recognition result has been recognized at the client device; and

providing the at least one recognition result to the client device.

17. The method of claim 16 , further comprising:

determining, in response in response to the server performing the second search for the second set of objects in the image data, that at least one recognition result has been recognized at the server; and

providing, from the server, the at least one recognition result to the client device.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2021
From: VERIZON MEDIA INC.
To: VERIZON PATENT AND LICENSING INC.
Reel/Frame 057453/0431 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: OATH INC.
To: VERIZON MEDIA INC.
Reel/Frame 054258/0635 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2018
From: YAHOO HOLDINGS, INC.
To: OATH INC.
Reel/Frame 045240/0310 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2017
From: YAHOO! INC.
To: YAHOO HOLDINGS, INC.
Reel/Frame 042963/0211 →
MERGER Recorded Sep 26, 2013
From: IQ ENGINES, INC.
To: YAHOO! INC.
Reel/Frame 031286/0221 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 29, 2012
From: GARRIGUES, PIERRE; CULPEPPER, BENJAMIN JACKSON; NGUYEN, HUY X.; CADIEU, CHARLES; STADLEN, ANDREW; PESAVENTO, GERRY; WARLAND, DAVID
To: IQ ENGINES INC.
Reel/Frame 028473/0046 →