IP Library Granted Patent US 11,544,509
Granted Patent B2
US 11,544,509 · App. 17/072,740 · Granted Jan 3, 2023

Methods, systems, articles of manufacture, and apparatus to classify labels based on images using artificial intelligence

Inventors: Roberto Arroyo (Guadalajara, ES); David Jiménez-Cabello (Guadalajara, ES); Javier Martínez Cebrián (Madrid, ES)
Assignee: Nielsen Consumer LLC
G06K9/6267G06K9/6201G06K9/6261G06K9/6289G06N3/08G06T3/40G06V10/25G06V2201/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,544,509
App. No.
17/072,740
Granted
Jan 3, 2023
Kind
B2
Abstract

Example methods, apparatus, and articles of manufacture to classify labels based on images using artificial intelligence are disclosed. An example apparatus includes a regional proposal network to determine a first bounding box for a first region of interest in a first input image of a product; and determine a second bounding box for a second region of interest in a second input image of the product; a neural network to: generate a first classification for a first label in the first input image using the first bounding box; and generate a second classification for a second label in the second input image using the second bounding box; a comparator to determine that the first input image and the second input image correspond to a same product; and a report generator to link the first classification and the second classification to the product.

Claims (42)

1. An apparatus comprising:

memory; and

processing circuitry to execute computer readable instructions to implement:

a regional proposal network to:

determine a first bounding box for a first region of interest in a first input image of a product; and

determine a second bounding box for a second region of interest in a second input image of the product;

a neural network to:

generate a first classification for a first label in the first input image using the first bounding box; and

generate a second classification for a second label in the second input image using the second bounding box;

a comparator to determine that the first input image and the second input image correspond to a same product; and

a report generator to link the first classification and the second classification to the product.

2. The apparatus of claim 1 , wherein the first input image and the second input image are crowdsourced images taken from computing devices of users.

3. The apparatus of claim 1 , wherein the comparator is to determine that the first input image and the second input image correspond to the same product based on at least one of (a) a first file name of the first input image and a second file name of the second input image or (b) first metadata of the first file name and second metadata of the second file name.

4. The apparatus of claim 1 , further including an image resizer to resize at least one of the first input image or the second input image to a preset size.

5. The apparatus of claim 1 , wherein the report generator is to generate a report corresponding to the product including the first label and the second label.

6. The apparatus of claim 1 , wherein the report generator is to generate a report corresponding to product and the linking of the first and second classifications.

7. The apparatus of claim 1 , wherein the second input image of the product does not include the second label.

8. A non-transitory computer readable storage medium comprising instructions which, when executed, cause one or more processors to at least:

determine a first bounding box for a first region of interest in a first input image of a product;

determine a second bounding box for a second region of interest in a second input image of the product;

generate a first classification for a first label in the first input image using the first bounding box;

generate a second classification for a second label in the second input image using the second bounding box;

determine that the first input image and the second input image correspond to a same product; and

link the first classification and the second classification to the product.

9. The non-transitory computer readable storage medium of claim 8 , wherein the first input image and the second input image are crowdsourced images taken from computing devices of users.

10. The non-transitory computer readable storage medium of claim 8 , wherein the instructions cause the one or more processors to determine that the first input image and the second input image correspond to the same product based on at least one of (a) a first file name of the first input image and a second file name of the second input image or (b) first metadata of the first file name and second metadata of the second file name.

11. The non-transitory computer readable storage medium of claim 8 , wherein the instructions cause the one or more processors to resize at least one of the first input image or the second input image to a preset size.

12. The non-transitory computer readable storage medium of claim 8 , wherein the instructions cause the one or more processors to generate a report corresponding to the product including the first label and the second label.

13. The non-transitory computer readable storage medium of claim 8 , wherein the instructions cause the one or more processors to generate a report corresponding to product and the linking of the first and second classifications.

14. The non-transitory computer readable storage medium of claim 8 , wherein the second input image of the product does not include the second label.

15. A method comprising:

determining, by executing an instruction with a processor, a first bounding box for a first region of interest in a first input image of a product;

determining, by executing an instruction with the processor, a second bounding box for a second region of interest in a second input image of the product;

generating, using a neural network, a first classification for a first label in the first input image using the first bounding box;

generating, using the neural network, a second classification for a second label in the second input image using the second bounding box;

determining, by executing an instruction with the processor, that the first input image and the second input image correspond to a same product; and

linking, by executing an instruction with the processor, the first classification and the second classification to the product.

16. The method of claim 15 , wherein the first input image and the second input image are crowdsourced images taken from computing devices of users.

17. The method of claim 15 , further including determining that the first input image and the second input image correspond to the same product based on at least one of (a) a first file name of the first input image and a second file name of the second input image or (b) first metadata of the first file name and second metadata of the second file name.

18. The method of claim 15 , further including resizing at least one of the first input image or the second input image to a preset size.

19. The method of claim 15 , further including generating a report corresponding to the product including the first label and the second label.

20. The method of claim 15 , further including generating a report corresponding to product and the linking of the first and second classifications.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNOR'S NAME INSIDE THE ASSIGNMENT DOCUMENT AND ON THE COVER SHEET PREVIOUSLY RECORDED AT REEL: 054380 FRAME: 0181. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Oct 26, 2022
From: JIMÉNEZ-CABELLO, DAVID
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 062320/0793 →
SECURITY INTEREST Recorded Mar 25, 2021
From: NIELSEN CONSUMER LLC; BYZZER INC.
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT AND COLLATERAL AGENT
Reel/Frame 055742/0719 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 17, 2021
From: THE NIELSEN COMPANY (US), LLC
To: NIELSEN CONSUMER LLC
Reel/Frame 055325/0353 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2020
From: ARROYO, ROBERTO; JIMÉNEZ, DAVID; CEBRIÁN, JAVIER MARTÍNEZ
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 054380/0181 →
Cited By (1)
US 12,561,950