IP Library Granted Patent US 9,595,098
Granted Patent B2
US 9,595,098 · App. 15/085,683 · Granted Mar 14, 2017

Image overlaying and comparison for inventory display auditing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,595,098
App. No.
15/085,683
Granted
Mar 14, 2017
Kind
B2
Abstract

Example methods disclosed herein to perform inventory display auditing include comparing a reference image of an inventory display obtained during a prior audit of the inventory display with a current image captured by a camera to determine whether the current image corresponds to the same inventory display as the reference image. Disclosed example methods also include, when the current image is determined to correspond to the same inventory display as the reference image, identifying a difference region in the current image corresponding to a portion of the inventory display that changed relative to the prior audit. Disclosed example methods further include, while the current image is displayed on a display screen, modifying a characteristic of the current image in the difference region to indicate a change in product inventory in the portion of the inventory display.

Claims (69)

1. A method to perform inventory display auditing, the method comprising:

comparing, by executing an instruction with a processor, a reference image of an inventory display obtained during a prior audit of the inventory display with a current image captured by a camera to determine whether the current image corresponds to the same inventory display as the reference image;

when the current image is determined to correspond to the same inventory display as the reference image, identifying, by executing an instruction with the processor, a difference region in the current image corresponding to a portion of the inventory display that changed relative to the prior audit; and

while the current image is displayed on a display screen, overlaying, by executing an instruction with the processor, an outline and text on the difference region in the current image to indicate a change in product inventory in the portion of the inventory display.

2. The method of claim 1 , further including:

segmenting the current image into a plurality of segmented regions;

segmenting the reference image into a plurality of segment regions; and

comparing the segmented regions of the current image to the segmented regions of the reference image to identify the difference region in the current image.

3. The method of claim 2 , wherein the segmented regions of the current image correspond to different shelves of the inventory display.

4. The method of claim 2 , wherein the segmented regions of the current image correspond to different sections of the inventory display.

5. The method of claim 2 , wherein the comparing of the segmented regions of the current image to the segmented regions of the reference image includes correlating ones of the segmented regions of the current image with respective ones of the segmented regions of the reference image having similar locations as the ones of the segmented regions of the current image within a tolerance.

6. A method to perform inventory display auditing, the method comprising:

comparing, by executing an instruction with a processor, a reference image of an inventory display obtained during a prior audit of the inventory display with a current image captured by a camera to determine whether the current image corresponds to the same inventory display as the reference image;

when the current image is determined to correspond to the same inventory display as the reference image, identifying, by executing an instruction with the processor, a difference region in the current image corresponding to a portion of the inventory display that changed relative to the prior audit by:

segmenting the current image into a plurality of segmented regions;

segmenting the reference image into a plurality of segment regions;

comparing the segmented regions of the current image to the segmented regions of the reference image to identify the difference region in the current image, wherein the comparing of the segmented regions of the current image to the segmented regions of the reference image includes correlating ones of the segmented regions of the current image with respective ones of the segmented regions of the reference image having similar locations as the ones of the segmented regions of the current image within a tolerance;

averaging correlation results determined from the correlating of the ones of the segmented regions of the current image with the respective ones of the segmented regions of the reference image to determine an average value; and

identifying a first one of the segmented regions of the current image as corresponding to the difference region when a first one of the correlations results determined from the correlating of the first one of the segmented regions of the current image with a respective first one of the segmented regions of the reference image exceeds the average value; and

while the current image is displayed on a display screen, modifying, by executing an instruction with the processor, a characteristic of the current image in the difference region to indicate a change in product inventory in the portion of the inventory display.

7. The method of claim 1 , further including modifying at least one of coloring or shading in the difference region of the current image while the current image is displayed on the display screen.

8. A tangible computer readable medium comprising computer readable instructions which, when executed, cause a processor to at least:

compare a reference image of an inventory display obtained during a prior audit of the inventory display with a current image captured by a camera to determine whether the current image corresponds to the same inventory display as the reference image;

when the current image is determined to correspond to the same inventory display as the reference image, identify a difference region in the current image corresponding to a portion of the inventory display that changed relative to the prior audit; and

while the current image is displayed on a display screen, overlay an outline and text on the difference region in the current image to indicate a change in product inventory in the portion of the inventory display.

9. The tangible computer readable medium of claim 8 , wherein the instructions, when executed, further cause the processor to:

segment the current image into a plurality of segmented regions;

segment the reference image into a plurality of segment regions; and

compare the segmented regions of the current image to the segmented regions of the reference image to identify the difference region in the current image.

10. The tangible computer readable medium of claim 9 , wherein the segmented regions of the current image correspond to different shelves of the inventory display.

11. The tangible computer readable medium of claim 9 , wherein the segmented regions of the current image correspond to different sections of the inventory display.

12. The tangible computer readable medium of claim 9 , wherein the instructions, when executed, cause the processor to compare the segmented regions of the current image to the segmented regions of the reference image by correlating ones of the segmented regions of the current image with respective ones of the segmented regions of the reference image having similar locations as the ones of the segmented regions of the current image within a tolerance.

13. A tangible computer readable medium comprising computer readable instructions which, when executed, further cause a processor to at least:

compare a reference image of an inventory display obtained during a prior audit of the inventory display with a current image captured by a camera to determine whether the current image corresponds to the same inventory display as the reference image;

when the current image is determined to correspond to the same inventory display as the reference image, identify a difference region in the current image corresponding to a portion of the inventory display that changed relative to the prior audit by causing the processor to:

segment the current image into a plurality of segmented regions;

segment the reference image into a plurality of segment regions;

compare the segmented regions of the current image to the segmented regions of the reference image to identify the difference region in the current image, wherein the instructions, when executed, cause the processor to compare the segmented regions of the current image to the segmented regions of the reference image by correlating ones of the segmented regions of the current image with respective ones of the segmented regions of the reference image having similar locations as the ones of the segmented regions of the current image within a tolerance;

average correlation results determined from the correlating of the ones of the segmented regions of the current image with the respective ones of the segmented regions of the reference image to determine an average value; and

identify a first one of the segmented regions of the current image as corresponding to the difference region when a first one of the correlations results determined from the correlating of the first one of the segmented regions of the current image with a respective first one of the segmented regions of the reference image exceeds the average value; and

while the current image is displayed on a display screen, modify a characteristic of the current image in the difference region to indicate a change in product inventory in the portion of the inventory display.

14. The tangible computer readable medium of claim 8 , wherein the instructions, when executed, further cause the processor to modify at least one of coloring or shading in the difference region of the current image while the current image is displayed on the display screen.

15. A portable device comprising:

memory to store a reference image of an inventory display obtained during a prior audit of the inventory display;

a camera;

a display screen; and

a comparator to:

compare the reference image with a current image captured by the camera to determine whether the current image corresponds to the same inventory display as the reference image;

when the current image is determined to correspond to the same inventory display as the reference image, identify a difference region in the current image corresponding to a portion of the inventory display that changed relative to the prior audit; and

while the current image is displayed on the display screen, overlay an outline and text on the difference region in the current image to indicate a change in product inventory in the portion of the inventory display.

16. The portable device of claim 15 , wherein the comparator is further to:

segment the current image into a plurality of segmented regions;

segment the reference image into a plurality of segment regions; and

compare the segmented regions of the current image to the segmented regions of the reference image to identify the difference region in the current image.

17. The portable device of claim 16 , wherein the segmented regions of the current image correspond to at least one of different shelves of the inventory display or different sections of the inventory display.

18. The portable device of claim 16 , wherein the comparator is to compare the segmented regions of the current image to the segmented regions of the reference image by correlating ones of the segmented regions of the current image with respective ones of the segmented regions of the reference image having similar locations as the ones of the segmented regions of the current image within a tolerance.

19. A portable device comprising:

a camera;

a display screen; and

a comparator to:

compare the reference image with a current image captured by the camera to determine whether the current image corresponds to the same inventory display as the reference image;

when the current image is determined to correspond to the same inventory display as the reference image, identify a difference region in the current image corresponding to a portion of the inventory display that changed relative to the prior audit, wherein to identify the difference region, the comparator is to:

segment the current image into a plurality of segmented regions;

segment the reference image into a plurality of segment regions;

compare the segmented regions of the current image to the segmented regions of the reference image to identify the difference region in the current image by correlating ones of the segmented regions of the current image with respective ones of the segmented regions of the reference image having similar locations as the ones of the segmented regions of the current image within a tolerance;

average correlation results determined from the correlating of the ones of the segmented regions of the current image with the respective ones of the segmented regions of the reference image to determine an average value; and

identify a first one of the segmented regions of the current image as corresponding to the difference region when a first one of the correlations results determined from the correlating of the first one of the segmented regions of the current image with a respective first one of the segmented regions of the reference image exceeds the average value; and

while the current image is displayed on the display screen, modify a characteristic of the current image in the difference region to indicate a change in product inventory in the portion of the inventory display.

20. The portable device of claim 15 , wherein the comparator is further to modify at least one of coloring or shading in the difference region of the current image while the current image is displayed on the display screen.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 22, 2016
From: HICKS, MICHAEL A.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 038348/0529 →