IP Library Granted Patent US 11,640,707
Granted Patent B2
US 11,640,707 · App. 17/201,909 · Granted May 2, 2023

Reducing scale estimate errors in shelf images

Inventors: Antonio Hurtado García (Valladolid, ES); Francisco Javier Martin (Valladolid, ES); Diego García Morate (Valladolid, ES)
Assignee: NIELSEN CONSUMER LLC
G06K9/00536G06F16/583G06T11/60G06V10/25G06V10/754G06V20/00G06V20/10G06Q10/087G06T2207/30128G06V20/68G06V2201/06G06V2201/09
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,640,707
App. No.
17/201,909
Granted
May 2, 2023
Kind
B2
Abstract

Example image processing methods, apparatus/systems and articles of manufacture are disclosed herein. An example apparatus includes an image recognition application to identify matches between stored patterns and objects detected in a shelf image, where the shelf image has a shelf image scale estimate. The example apparatus further includes a scale corrector to calculate deviation values between sizes of (A) a first set of the objects detected in the shelf image and (B) a first set of the stored patterns matched with the first set of the objects and reduce an error of the shelf image scale estimate by calculating a scale correction value for the shelf image scale estimate based on the deviation values.

Claims (35)

1. An apparatus comprising:

memory; and

at least one processor to:

determine optimized scale values for a first set of patterns of a plurality of patterns stored in a database, the optimized scale values determined based on scale estimates associated with the first set of patterns;

generate rescaled versions of the first set of patterns using the optimized scale values for the first set of patterns and based on a shelf image scale estimate of a shelf image; and

identify, using the rescaled versions of the first set of patterns, matches between the first set of patterns and objects appearing in the shelf image.

2. The apparatus of claim 1 , wherein the at least one processor is to, prior to determining the optimized scale values, determine that each of the first set of patterns has a number of scale estimates that satisfy a threshold.

3. The apparatus of claim 2 , wherein the plurality of patterns includes a second set of patterns having a number of scale estimates that do not satisfy the threshold.

4. The apparatus of claim 1 , wherein the optimized scale values are a median of the scale estimates associated with respective ones of the first set of patterns.

5. The apparatus of claim 1 , wherein the at least one processor is to determine deviation values between sizes of the rescaled versions of the first set of patterns and the objects matched with the first set of patterns.

6. The apparatus of claim 5 , wherein the at least one processor is to determine a scale correction value for the shelf image scale estimate based on the deviation values, the scale correction value to be used during a subsequent analysis of the shelf image.

7. The apparatus of claim 6 , wherein the at least one processor is to determine a new scale estimate for a first pattern of the first set of patterns based on a size of an object matched with a rescaled version of the first pattern and the scale correction value.

8. An apparatus comprising:

a memory; and

at least one processor to:

analyze a shelf image a first time to identify matches between patterns stored in a database and objects appearing in the shelf image based on a first shelf image scale estimate of the shelf image;

determine a second shelf image scale estimate for the shelf image based on deviation values between (a) sizes of a first set of objects detected in the shelf image and (b) a first set of the patterns matched with the first set of objects; and

analyze the shelf image a second time to identify matches between the patterns stored in the database and the objects appearing in the shelf image based on the second shelf image scale estimate.

9. The apparatus of claim 8 , wherein the at least one processor is to determine the deviation values by determining a difference in pixel dimension of a first scaled version of a first pattern of the first set of the patterns and a first object of the first set of objects match with the first pattern.

10. The apparatus of claim 9 , wherein the at least one processor is to, prior to analyzing the shelf image the first time, generate the first scaled version of the first pattern by rescaling the first pattern based on the first shelf image scale estimate.

11. The apparatus of claim 8 , wherein the at least one processor is to, prior to analyzing the shelf image the first time, rescale the patterns to the first shelf image scale estimate.

12. The apparatus of claim 8 , wherein the first set of the patterns have optimized scale values, and wherein the at least one processor is to, prior to analyzing the shelf image the first time, determine the optimized scale values for the first set of the patterns.

13. The apparatus of claim 12 , wherein the optimized scale values for the first set of the patterns are based on arrays of scale estimates for respective ones of the first set of the patterns, wherein the first set of the patterns have a number of scale estimates that satisfy a threshold, and wherein a second set of the patterns have a number of scale estimates that do not satisfy the threshold.

14. An apparatus comprising:

memory; and

at least one processor to:

identify matches between stored patterns and objects detected in a shelf image, the shelf image having a shelf image scale estimate;

calculate deviation values between sizes of (A) a first set of the objects detected in the shelf image and (B) a first set of the stored patterns matched with the first set of the objects; and

reduce an error of the shelf image scale estimate by calculating a scale correction value for the shelf image scale estimate based on the deviation values.

15. The apparatus of claim 14 , wherein the at least one processor is to calculate the deviation values by determining a difference in pixel dimensions of a first scaled version of a first pattern of the first set of the stored patterns and a first object of the first set of the objects matched with the first pattern.

16. The apparatus of claim 15 , wherein the at least one processor is to generate the first scaled version of the first pattern by rescaling the first pattern based on the shelf image scale estimate.

17. The apparatus of claim 14 , wherein the first set of the stored patterns have optimized scales.

18. The apparatus of claim 17 , wherein the at least one processor is to determine the optimized scales for the first set of the stored patterns.

19. The apparatus of claim 17 , wherein the optimized scales are a median of estimated scales for the first set of the stored patterns.

20. The apparatus of claim 14 , wherein the stored patterns include a second set of the stored patterns, the first set of the stored patterns having a number of scale estimates that satisfy a threshold, the second set of the stored patterns having a number of scale estimates that do not satisfy the threshold.

Assignments (3)
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Dec 16, 2022
From: NIELSEN CONSUMER LLC
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 062142/0346 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 26, 2021
From: GARCÍA, ANTONIO HURTADO; MARTIN, FRANCISCO JAVIER; MORATE, DIEGO GARCÍA
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 056034/0007 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 26, 2021
From: THE NIELSEN COMPANY (US), LLC
To: NIELSEN CONSUMER LLC
Reel/Frame 056034/0063 →