IP Library Granted Patent US 8,917,902
Granted Patent B2
US 8,917,902 · App. 13/217,031 · Granted Dec 23, 2014

Image overlaying and comparison for inventory display auditing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,917,902
App. No.
13/217,031
Granted
Dec 23, 2014
Kind
B2
Abstract

Image overlaying and comparison for inventory display auditing is disclosed herein. An example method to perform inventory display auditing disclosed herein comprises overlaying a reference image over a current image displayed on a camera display, the reference image corresponding to an inventory display to be audited, comparing the reference image and the current image to determine whether the current image and the reference image correspond to a same scene and when the reference image and the current image are determined to correspond to the same scene, indicating a difference region in the current image displayed on the camera display, the difference region being a first region of the current image that differs from a corresponding first region of the reference image.

Claims (48)

1. A method to perform inventory display auditing, the method comprising:

overlaying a reference image over a current image displayed on a camera display, the reference image corresponding to an inventory display to be audited;

comparing the reference image and the current image to determine whether the current image and the reference image correspond to a same scene; and

when the reference image and the current image are determined to correspond to the same scene, modifying a characteristic of the current image in a difference region identified in the current image to display the difference region in an overlay of the reference image and the current image displayed on the camera display, the difference region being a first region of the current image that differs from a corresponding first region of the reference image.

2. A method as defined in claim 1 wherein overlaying the reference image comprises:

combining a ghost version of the reference image with the current image to determine a combined image; and

displaying the combined image on the camera display.

3. A method as defined in claim 1 wherein comparing the reference image and the current image comprises correlating the reference image and the current image.

4. A method as defined in claim 3 wherein the reference image and the current image are determined to correspond to the same scene when a result of correlating the reference image and the current image satisfies a threshold.

5. A method as defined in claim 1 further comprising, when the reference image and the current image are determined to correspond to the same scene, comparing segmented regions of the current image and segmented regions of the reference image to determine the difference region.

6. A method as defined in claim 1 wherein the current image is a first captured image obtained from a camera corresponding to the camera display, and further comprising, when the reference image and the current image are determined to not correspond to the same scene:

determining a directional difference between the first captured image and the reference image; and

presenting a directional prompt on the camera display, the directional prompt to indicate a direction in which to move the camera to increase a likelihood that a second captured image and the reference image will be determined to correspond to the same scene.

7. A method as defined in claim 1 wherein the current image is a first captured image obtained from a camera corresponding to the camera display, and further comprising tracking the difference region in subsequent captured images obtained from the camera.

8. A tangible computer readable medium comprising machine readable instructions which, when executed, cause a computing device to at least:

overlay a reference image over a first captured image displayed on an imaging display, the reference image corresponding to an inventory display to be audited, the reference image taken before the first captured image;

compare the reference image and the first captured image to determine whether the first captured image and the reference image correspond to a same scene;

when the reference image and the first captured image are determined to correspond to the same scene, indicate a difference region in the first captured image, the difference region being a first region of the first captured image that differs from a corresponding first region of the reference image; and

when the reference image and the first captured image are determined to not correspond to the same scene:

determine a directional difference between the first captured image and the reference image; and

present a directional prompt on the imaging display, the directional prompt to indicate a direction in which to move the computing device to increase a likelihood that a second captured image and the reference image will be determined to correspond to the same scene.

9. A tangible computer readable medium as defined in claim 8 wherein the machine readable instructions, when executed, further cause the computing device to:

combine a ghost version of the reference image with the first captured image to determine a combined image; and

display the combined image on the imaging display.

10. A tangible computer readable medium as defined in claim 8 wherein the machine readable instructions, when executed, further cause the computing device to correlate the reference image and the first captured image to compare the reference image and the first captured image.

11. A tangible computer readable medium as defined in claim 10 wherein the reference image and the first captured image are determined to correspond to the same scene when a result of correlating the reference image and the first captured image satisfies a threshold.

12. A tangible computer readable medium as defined in claim 8 wherein the machine readable instructions, when executed, further cause the computing device to:

modify an image characteristic of the first region of the first captured image to determine a modified captured image; and

display the modified captured image.

13. A tangible computer readable medium as defined in claim 8 wherein the machine readable instructions, when executed, cause the computing device to compare segmented regions of the first captured image and segmented regions of the reference image to determine the difference region.

14. A tangible computer readable medium as defined in claim 8 wherein the machine readable instructions, when executed, further cause the computing device to track the difference region in subsequent captured images obtained from the computing device.

15. A method as defined in claim 1 wherein modifying the characteristic of the current image in the difference region comprises modifying at least one of a luminance or a chrominance of the current image in the difference region.

16. A method as defined in claim 5 further comprising:

determining an average value of correlation results obtained by correlating respective ones of the segmented regions of the current image with corresponding ones of the segmented regions of the reference image, the correlation results including a first correlation result obtained by correlating the first region of the current image with the corresponding first region of the reference image; and

comparing the first correlation result with the average value to identify the difference region in the current image.

17. A tangible computer readable medium as defined in claim 8 wherein the machine readable instructions, when executed, further cause the computing device to correlate the first captured image with shifted versions of the reference image to determine the directional difference between the first captured image and the reference image.

18. A tangible computer readable medium as defined in claim 17 wherein the machine readable instructions, when executed, further cause the computing device to:

determine the directional difference to correspond to a first positional shift of the reference image that increases a correlation between the first captured image and the reference image; and

translate the first positional shift into a positional movement of the computing device that would cause the second captured image and the reference image to exhibit the first positional shift, the directional prompt presented on the imaging display being representative of the positional movement.

19. An apparatus comprising:

a device display;

a positioner to:

overlay a reference image over a first captured image displayed on the device display, the reference image corresponding to an inventory display to be audited; and

compare the reference image and the first captured image to determine whether the first captured image and the reference image correspond to a same scene; and

a comparator to, in response to the reference image and the first captured image being determined by the positioner to correspond to the same scene, modify a characteristic of the first captured image in a difference region identified in the first captured image to display the difference region in an overlay of the reference image and the first captured image displayed on the device display, the difference region being a first region of the first captured image that differs from a corresponding first region of the reference image.

20. An apparatus as defined in claim 19 wherein the comparator is further to, in response to the reference image and the first captured image being determined by the positioner to not correspond to the same scene:

determine a directional difference between the first captured image and the reference image; and

present a directional prompt on the imaging display, the directional prompt to indicate a direction in which to move the apparatus to increase a likelihood that a second captured image and the reference image will be determined to correspond to the same scene.

Assignments (10)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
RELEASE (REEL 037172 / FRAME 0415) Recorded Oct 13, 2022
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 061750/0221 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
SUPPLEMENTAL IP SECURITY AGREEMENT Recorded Nov 30, 2015
From: THE NIELSEN COMPANY ((US), LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT FOR THE FIRST LIEN SECURED PARTIES
Reel/Frame 037172/0415 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2011
From: HICKS, MICHAEL A.
To: THE NIELSEN COMPANY (US), LLC, A DELAWARE LIMITED LIABILITY COMPANY
Reel/Frame 026984/0035 →