IP Library Granted Patent US 12,183,049
Granted Patent B2
US 12,183,049 · App. 18/492,532 · Granted Dec 31, 2024

Methods, systems, articles of manufacture, and apparatus to classify labels based on images using artificial intelligence

Inventors: Roberto Arroyo (Guadalajara, ES); David Jiménez-Cabello (Guadalajara, ES); Javier Martínez Cebrián (Madrid, ES)
Assignee: Nielsen Consumer LLC
G06V10/25G06F18/2163G06F18/22G06F18/24G06F18/251G06N3/08G06T3/40G06V10/761G06V10/764G06V10/771G06V10/809G06V10/82G06V20/70G06V2201/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,183,049
App. No.
18/492,532
Granted
Dec 31, 2024
Kind
B2
Abstract

Example methods, apparatus, and articles of manufacture to classify labels based on images using artificial intelligence are disclosed. An example apparatus includes a regional proposal network to determine a first bounding box for a first region of interest in a first input image of a product; and determine a second bounding box for a second region of interest in a second input image of the product; a neural network to: generate a first classification for a first label in the first input image using the first bounding box; and generate a second classification for a second label in the second input image using the second bounding box; a comparator to determine that the first input image and the second input image correspond to a same product; and a report generator to link the first classification and the second classification to the product.

Claims (61)

1. An apparatus comprising:

interface circuitry to receive first and second images corresponding to a product;

machine readable instructions; and

at least one processor circuit to execute the machine readable instructions to at least:

detect a first label in the first image, the first label defined by a first bounding box;

classify the first label based on the first bounding box;

detect a second label in the second image, the second label defined by a second bounding box;

classify the second label based on the second bounding box; and

when the first classification and the second classification are associated with the product, assign the first and second classifications to the product.

2. The apparatus of claim 1 , wherein the first image corresponds to a first portion of the product and the second image corresponds to a second portion of the product, the first portion to be at least partially offset relative to the second portion.

3. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to detect the first label by executing the machine readable instructions to:

extract a feature map from the first image, the feature map including points; and

apply a region proposal network to the feature map to:

generate, at respective ones of the points in the feature map, a set of anchor boxes based on predetermined anchor box sizes and anchor box ratios;

identify ones of the anchor boxes that include the first label, the ones of the anchor boxes to include a respective confidence score;

determine bounding box coordinates for the ones of the anchor boxes, the bounding box coordinates corresponding to positions relative to the first image; and

identify the first bounding box as including the label by applying a non-maximum selection technique to the ones of the anchor boxes.

4. The apparatus of claim 3 , wherein the one or more of the at least one processor circuit is to execute the machine readable instructions to train the region proposal network by configuring a first hyperparameter corresponding to the predetermined anchor box ratios and a second hyperparameter corresponding to the predetermined anchor box sizes.

5. The apparatus of claim 4 , wherein the one or more of the at least one processor circuit is to execute the machine readable instructions to configure the first hyperparameter to anchor box ratios of (a) 1:2, (b) 1:1, and (c) 2:1.

6. The apparatus of claim 4 , wherein the one or more of the at least one processor circuit is to execute the machine readable instructions to configure the second hyperparameter to anchor box scales of 2, 4, and 6.

7. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to execute the machine readable instructions to:

detect a third label in the second image, the third label defined by a third bounding box;

classify the third label based on the third bounding box; and

assign the third classification to the product.

8. A non-transitory machine readable storage medium comprising instructions to cause at least one processor circuit to at least:

detect a first label in a first image, the first label defined by a first bounding box, the first image to correspond to a first product;

classify the first label based on the first bounding box;

detect a second label in a second image, the second label defined by a second bounding box, the second image to correspond to the first product;

classify the second label based on the second bounding box; and

when the first classification and the second classification are associated with the product, assign the first and second classifications to the product.

9. The non-transitory machine readable storage medium of claim 8 , wherein the first image corresponds to a first region of the product and the second image corresponds to a second region of the product, the first region to be at least partially different than the second region.

10. The non-transitory machine readable storage medium of claim 8 , wherein the instructions cause one or more of the at least one processor circuit to:

extract a feature map from the first image, the feature map including points;

generate, by applying a region proposal network to the feature map, a set of anchors boxes at respective ones of the points in the feature map based on predetermined anchor box sizes and anchor box ratios;

identify ones of the anchor boxes that include the first label, the ones of the anchor boxes to include a respective confidence score;

determine bounding box coordinates for the ones of the anchor boxes, the bounding box coordinates corresponding to positions relative to the first image; and

identify the first bounding box as including the first label by applying a non-maximum selection technique to the ones of the anchor boxes.

11. The non-transitory machine readable storage medium of claim 10 , wherein the instructions cause the one or more of the at least one processor circuit to train the region proposal network by configuring a first hyperparameter corresponding to the predetermined anchor box ratios and a second hyperparameter corresponding to the predetermined anchor box sizes.

12. The non-transitory machine readable storage medium of claim 11 , wherein the instructions are to cause the one or more of the at least one processor circuit to configure the first hyperparameter to anchor box ratios of (a) 1:2, (b) 1:1, and (c) 2:1.

13. The non-transitory machine readable storage medium of claim 11 , wherein the instructions are to cause the one or more of the at least one processor circuit to configure the second hyperparameter to anchor box scales of 2, 4, and 6.

14. The non-transitory machine readable storage medium of claim 8 , wherein the instructions are to cause the one or more of the at least one processor circuit to:

detect a third label in the second image, the third label defined by a third bounding box;

classify the third label based on the third bounding box; and

assign the third classification to the product.

15. A method comprising:

detecting, by executing a machine readable instruction with programmable circuitry, a first label in a first image, the first label defined by a first bounding box, the first image corresponding to a product;

classifying, by executing a machine readable instruction with the programmable circuitry, the first label based on the first bounding box;

detecting, by executing a machine readable instruction with the programmable circuitry, a second label in a second image, the second label defined by a second bounding box the second image corresponding to the product;

classifying, by executing a machine readable instruction with the programmable circuitry, the second label based on the second bounding box; and

when the first classification and the second classification are associated with the product, assigning, by executing a machine readable instruction with the programmable circuitry, the first and second classifications to the product.

16. The method of claim 15 , wherein the first image corresponds to a first portion of the product and the second image corresponds to a second portion of the product, the second portion to be at least partially offset from the first portion of the product.

17. The method of claim 15 , wherein the detecting of the first label includes:

extracting a feature map from the first image, the feature map including points; and

applying a region proposal network to the feature map, wherein the applying of the region proposal network to includes:

generating, at respective ones of the points in the feature map, a set of anchors boxes based on predetermined anchor box sizes and anchor box ratios;

identifying ones of the anchor boxes that include the first label, the ones of the anchor boxes to include a respective confidence score;

determining bounding box coordinates for the ones of the anchor boxes, the bounding box coordinates corresponding to positions relative to the first image; and

identifying the first bounding box as including the label by applying a non-maximum selection technique to the ones of the anchor boxes.

18. The method of claim 17 , further including training the region proposal network by configuring a first hyperparameter corresponding to the predetermined anchor box ratios and a second hyperparameter corresponding to the predetermined anchor box sizes.

19. The method of claim 18 , wherein the configuring of the first hyperparameter includes tuning the first hyperparameter to generate anchor box ratios of (a) 1:2, (b) 1:1, and (c) 2:1.

20. The method of claim 18 , wherein the configuring of the second hyperparameter includes tuning the second hyperparameter to generate anchor box scales of 2, 4, and 6.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 24, 2024
From: ARROYO, ROBERTO; JIMÉNEZ-CABELLO, DAVID; MARTÍNEZ CEBRIÁN, JAVIER
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 067211/0156 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 24, 2024
From: THE NIELSEN COMPANY (US), LLC
To: NIELSEN CONSUMER LLC
Reel/Frame 067211/0167 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jan 19, 2024
From: NIELSEN CONSUMER LLC
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AND COLLATERAL AGENT
Reel/Frame 066355/0213 →
Continuity (5)
Continuation 18148131 · Dec 29, 2022
Continuation 17072740 · Oct 16, 2020
Provisional Application 63046363 · Jun 30, 2020
Related Publication 20240135669A1 · Apr 25, 2024
Related Publication 20240233309A9 · Jul 11, 2024