IP Library Granted Patent US 9,710,723
Granted Patent B2
US 9,710,723 · App. 15/417,864 · Granted Jul 18, 2017

Context-based image recognition for consumer market research

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,710,723
App. No.
15/417,864
Granted
Jul 18, 2017
Kind
B2
Abstract

Example methods, apparatus, systems to perform context-based image recognition are disclosed. Disclosed example methods include comparing a first and second reference region of a reference image with a first and second region of a first captured image to determine a first and second match value. In response to determining the first and second match values satisfy a first and second threshold: associating the first captured image with a product associated with the first reference image, the product being associated with a plurality of second reference images stored in the database. Decrementing index values associated with ones of the second reference images. Removing a reference record associated with one of the second reference images from the database based on a corresponding one of the second index values satisfying a first condition, the removal of the reference record reducing the storage space required to store reference records in the database.

Claims (61)

1. An apparatus, comprising:

a reference comparator to:

compare a first reference region of a first reference image stored in a database with a first region of a first image to determine a first match value, the first region corresponding spatially to the first reference region;

in response to determining the first match value satisfies a first threshold, compare a second reference region of the first reference image with a second region of the first image to determine a second match value, the second region corresponding spatially to the second reference region; and

in response to determining the second match value satisfies a second threshold, associate the first image with a product associated with the first reference image, the product being associated with a plurality of second reference images stored in the database; and

a reference image manager to, in response to determining the second match value satisfies the second threshold:

decrement second index values associated with ones of the second reference images; and

remove a reference record associated with one of the second reference images from the database based on a corresponding one of the second index values satisfying a first condition, the removal of the reference record reducing a storage space required to store reference records in the database, at least one of the reference comparator or the reference image manager including hardware.

2. The apparatus as defined in claim 1 , wherein the first reference region has a first significance value and the second reference region has a second significance value and the first significance value is higher than the second significance value.

3. The apparatus as defined in claim 1 , wherein the first condition is satisfied when the one of the second index values does not satisfy a third threshold.

4. The apparatus as defined in claim 1 , further including:

in response to determining the second match value satisfies the second threshold:

the reference comparator to designate the first reference image as a match to the first image; and

the reference image manager to select the second index values to decrement based on corresponding ones of the second reference images not being designated as a match to the first image.

5. The apparatus as defined in claim 4 , wherein the reference comparator is further to:

before the reference image manager decrements one of the second index values, for a corresponding one of the second reference images:

compare a third reference region of the corresponding second reference image with the first region of the first image to determine a third match value, the first region corresponding spatially to the third reference region;

in response to determining the third match value satisfies a fourth threshold, compare a fourth reference region of the corresponding second reference image with the second region of the first image to determine a fourth match value, the second region corresponding spatially to the fourth reference region; and

in response to determining the fourth match value satisfies a fifth threshold, increment a corresponding one of the index values associated with the corresponding second reference image, and designate the corresponding second reference image as a match to the first image.

6. The apparatus as defined in claim 1 , wherein the first condition is satisfied when the one of the second index values is a lowest index value associated with the product.

7. The apparatus as defined in claim 1 , wherein a maximum number of reference images stored in the database that are associated with the product is limited to a predetermined number.

8. A tangible computer readable storage medium comprising instructions which, when executed, cause a machine to at least:

compare a first reference region of a first reference image stored in a database with a first region of a first image to determine a first match value, the first region corresponding spatially to the first reference region;

in response to determining the first match value satisfies a first threshold, compare a second reference region of the first reference image with a second region of the first image to determine a second match value, the second region corresponding spatially to the second reference region; and

in response to determining the second match value satisfies a second threshold:

associate the first image with a product associated with the first reference image, the product being associated with a plurality of second reference images stored in the database;

decrement second index values associated with ones of the second reference images; and

remove a reference record associated with one of the second reference images from the database based on a corresponding one of the second index values satisfying a first condition, the removal of the reference record reducing a storage space required to store reference records in the database.

9. The tangible computer readable medium as defined in claim 8 , wherein the first reference region has a first significance value and the second reference region has a second significance value and the first significance value is higher than the second significance value.

10. The tangible computer readable medium as defined in claim 8 , wherein the first condition is satisfied when the one of the second index values does not satisfy a third threshold.

11. The tangible computer readable medium as defined in claim 8 , wherein the instruction, when executed, further cause the machine to:

in response to determining the second match value satisfies the second threshold:

designate the first reference image as a match to the first image; and

select the second index values to decrement based on corresponding ones of the second reference images not being designated as a match to the first image.

12. The tangible computer readable medium as defined in claim 11 , wherein the instruction, when executed, further cause the machine to:

before decrementing one of the second index values, for a corresponding one of the second reference images:

compare a third reference region of the corresponding second reference image with the first region of the first image to determine a third match value, the first region corresponding spatially to the third reference region;

in response to determining the third match value satisfies a fourth threshold, compare a fourth reference region of the corresponding second reference image with the second region of the first image to determine a fourth match value, the second region corresponding spatially to the fourth reference region; and

in response to determining the fourth match value satisfies a fifth threshold, increment a corresponding one of the index values associated with the corresponding second reference image, and designate the corresponding second reference image as a match to the first image.

13. The tangible computer readable medium as defined in claim 8 , wherein the first condition is satisfied when the one of the second index values is a lowest index value associated with the product.

14. The tangible computer readable medium as defined in claim 8 , wherein a maximum number of reference images stored in the database that are associated with the product is limited to a predetermined number.

15. A method, comprising:

receiving a first image, the first image captured by a mobile device;

comparing, by executing an instruction with a processor, a first reference region of a first reference image stored in a database with a first region of the first image to determine a first match value, the first region corresponding spatially to the first reference region;

in response to determining the first match value satisfies a first threshold, comparing, by executing an instruction with the processor, a second reference region of the first reference image with a second region of the first image to determine a second match value, the second region corresponding spatially to the second reference region; and

in response to determining the second match value satisfies a second threshold:

associating, by executing an instruction with the processor, the first image with a product associated with the first reference image, the product being associated with a plurality of second reference images stored in the database;

decrementing, by executing an instruction with the processor, second index values associated with ones of the second reference images; and

removing, by executing an instruction with the processor, a reference record associated with one of the second reference images from the database based on a corresponding one of the second index values satisfying a first condition, the removal of the reference record reducing a storage space required to store reference records in the database.

16. The method as defined in claim 15 , wherein the first reference region has a first significance value and the second reference region has a second significance value and the first significance value is higher than the second significance value.

17. The method as defined in claim 15 , wherein the first condition is satisfied when the one of the second index values does not satisfy a third threshold.

18. The method as defined in claim 15 , wherein the first condition is satisfied when the one of the second index values is a lowest index value associated with the product.

19. The method as defined in claim 15 , further including:

in response to determining the second match value satisfies the second threshold:

designating the first reference image as a match to the first image; and

selecting the second index values to decrement based on corresponding ones of the second reference images not being designated as a match to the first image.

20. The method as defined in claim 19 , further including:

before decrementing one of the second index values, for a corresponding one of the second reference images:

comparing, by executing an instruction with the processor, a third reference region of the corresponding second reference image with the first region of the first image to determine a third match value, the first region corresponding spatially to the third reference region;

in response to determining the third match value satisfies a fourth threshold, comparing, by executing an instruction with the processor, a fourth reference region of the corresponding second reference image with the second region of the first image to determine a fourth match value, the second region corresponding spatially to the fourth reference region; and

in response to determining the fourth match value satisfies a fifth threshold, incrementing, by executing an instruction with the processor, a corresponding one of the index values associated with the corresponding second reference image, and designating the corresponding second reference image as a match to the first image.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 31, 2017
From: XIE, WEI; BIVINS, MICHAEL ALLEN; JONES, STEVE; DENG, KEVIN KEQIANG; SCHILLER, BRIAN; SULLIVAN, JONATHAN; TERRAZAS, ALEJANDRO; SHEPPARD, MICHAEL
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 041131/0682 →