IP Library Granted Patent US 11,853,347
Granted Patent B2
US 11,853,347 · App. 17/892,679 · Granted Dec 26, 2023

Product auditing in point-of-sale images

Inventors: Diego García Morate (Valladolid, ES); Antonio Hurtado García (Valladolid, ES); Jorge Martin Martin (Valladolid, ES); Miguel Garcia Carranza (Valladolid, ES)
Assignee: Nielsen Consumer, LLC
G06F16/51G06F16/5854G06Q10/087
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,853,347
App. No.
17/892,679
Granted
Dec 26, 2023
Kind
B2
Abstract

Examples methods, apparatus/systems and articles of manufacture for auditing point-of-sale images are disclosed herein. Example methods disclosed herein include comparing a region of interest of an image displayed via a user interface with a plurality of reference product images stored in a database to identify a plurality of candidate product images from the plurality of reference product images as potential matches to a first product depicted in the image. For example, the candidate product images are associated with respective confidence levels indicating respective likelihoods of matching the first product. Disclosed example methods also include displaying, via the user interface, the candidate product images simultaneously with the image in a manner based on the respective confidence levels, and automatically selecting a first one of the candidate product images as matching the first product based on the respective confidence levels.

Claims (49)

1. An apparatus comprising:

a user interface;

machine readable instructions; and

processor circuitry to execute the machine readable instructions to:

cause the user interface to display an image depicting products;

identify a region of interest in the image having a first one of the products;

compare the region of interest with reference product images stored in a database to identify candidate product images from the reference product images as potential matches to the first one of the products in the region of interest;

cause the user interface to display a frame overlaying the image around the region of interest;

in response to user selection of the frame, cause the user interface to display a window over a section of the image adjacent to the region of interest, the window including the candidate product images; and

in response to user selection of one of the candidate product images in the window, update a record of products associated with the image.

2. The apparatus of claim 1 , wherein the processor circuitry is to assign confidence scores to the candidate product images representative of likelihoods of matching the first one of the products, and wherein the processor circuitry is to cause the user interface to display the candidate product images in order of the confidence scores.

3. The apparatus of claim 2 , wherein the processor circuitry is to cause the user interface to display the confidence scores in the window adjacent respective ones of the candidate product images.

4. The apparatus of claim 2 , wherein the processor circuitry is to:

compare confidence scores of the reference product images to a threshold; and

cause the user interface to display the reference product images with confidence scores that satisfy the threshold.

5. The apparatus of claim 1 , wherein the processor circuitry is to identify the region of interest based on user input highlighting the region of interest on the user interface.

6. The apparatus of claim 1 , wherein the processor circuitry is to, in response to the user selection of the one of the candidate product images in the window, highlight a section of the window around the one of the candidate product images in the window.

7. The apparatus of claim 1 , wherein the processor circuitry is to cause the user interface to display a product information section in the window with information corresponding to the one of the candidate product images, the product information section including an enlarged version of the one of the candidate product images corresponding to the user selection.

8. A non-transitory machine readable storage medium comprising instructions that cause a machine to at least:

display a point-of-sale image;

identify a region of interest in the point-of-sale image;

compare the region of interest with a plurality of reference product images to identify candidate product images from the plurality of reference product images as potential matches to a first product in the region of interest;

display a frame overlaying the image around the region of interest;

in response to user selection of the frame, display a window over a section of the image adjacent to the region of interest, the window including the candidate product images; and

in response to user selection of one of the candidate product images in the window, update a record of products associated with the point-of-sale image.

9. The non-transitory machine readable storage medium of claim 8 , wherein the instructions cause the machine to:

assign confidence scores to the candidate product images representative of likelihoods of matching the first product; and

display the candidate product images in order of the confidence scores.

10. The non-transitory machine readable storage medium of claim 9 , wherein the instructions cause the machine to display the confidence scores in the window adjacent the respective candidate product images.

11. The non-transitory machine readable storage medium of claim 9 , wherein the instructions cause the machine to:

compare confidence scores of the reference product images to a threshold; and

display the reference product images with confidence scores that satisfy the threshold.

12. The non-transitory machine readable storage medium of claim 8 , wherein the instructions cause the machine to, in response to the user selection of the one of the candidate product images in the window, highlight a section of the window around the one of the candidate product images in the window.

13. The non-transitory machine readable storage medium of claim 8 , wherein the instructions cause the machine to display a product information section in the window with information corresponding to the one of the candidate product images, the product information section including an enlarged version of the one of the candidate product images corresponding to the user selection.

14. A method comprising:

displaying, via a user interface, an image depicting products;

identifying a region of interest in the image having a first product;

comparing the region of interest with reference product images stored in a database to identify candidate product images from the reference product images as potential matches to the first product in the region of interest;

displaying, via the user interface, a frame overlaying the image around the region of interest;

in response to user selection of the frame, displaying, via the user interface, a window over a section of the image adjacent to the region of interest, the window including the candidate product images; and

in response to user selection of one of the candidate product images in the window, updating a record of products associated with the image.

15. The method of claim 14 , further including:

assigning confidence scores to the candidate product images representative of likelihoods of matching the first product; and

displaying, via the user interface, the candidate product images in order of the confidence scores.

16. The method of claim 15 , further including displaying, via the user interface, the confidence scores in the window adjacent the respective candidate product images.

17. The method of claim 15 , further including:

comparing confidence scores of the reference product images to a threshold; and

displaying, via the user interface, the reference product images with confidence scores that satisfy the threshold.

18. The method of claim 14 , further including, in response to the user selection of the one of the candidate product images in the window, highlighting a section of the window around the one of the candidate product images in the window.

Assignments (4)
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jun 20, 2024
From: NIELSEN CONSUMER LLC
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AND COLLATERAL AGENT
Reel/Frame 067792/0978 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Dec 16, 2022
From: NIELSEN CONSUMER LLC
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 062142/0346 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2022
From: THE NIELSEN COMPANY (US), LLC
To: NIELSEN CONSUMER LLC
Reel/Frame 062037/0004 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 8, 2022
From: MORATE, DIEGO GARCÍA; GARCÍA, ANTONIO HURTADO; MARTIN, JORGE MARTIN; CARRANZA, MIGUEL GARCIA
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 062095/0589 →