IP Library Granted Patent US 11,544,508
Granted Patent B2
US 11,544,508 · App. 17/102,064 · Granted Jan 3, 2023

Methods, systems, articles of manufacture, and apparatus to recalibrate confidences for image classification

Inventors: Emilio Almazan (Madrid, ES); Aitor Aller Beascoechea (Madrid, ES); Javier Tovar Velasco (Valladolid, ES)
Assignee: Nielsen Consumer LLC
G06K9/6262G06K9/6256G06K9/6277G06N3/04G06N3/08G06V10/23G06V10/267G06V10/426G06Q10/087
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,544,508
App. No.
17/102,064
Granted
Jan 3, 2023
Kind
B2
Abstract

Methods, systems, articles of manufacture, and apparatus to recalibrate confidences for image classification are disclosed. An example apparatus to classify an image includes an image crop detector to detect a first image crop from the image, the first image crop corresponding to a first object, a grouping controller to select a second image crop corresponding to a second object at a location of the first object, a prediction generator to, in response to executing a trained model, determine a label corresponding to the first object and a confidence level associated with the label, and a confidence recalibrator to recalibrate the confidence level based on a probability of the first object having a first attribute based on the second object having a second attribute, the confidence level recalibrated to increase an accuracy of the image classification.

Claims (33)

1. An apparatus to classify an image, the apparatus comprising:

an image crop detector to detect a first image crop from the image, the first image crop corresponding to a first object;

a grouping controller to select a second image crop corresponding to a second object proximate a location of the first object;

a prediction generator to, in response to executing a trained model, determine (a) a label corresponding to the first object and (b) a confidence level associated with the label; and

a confidence recalibrator to recalibrate the confidence level by calculating a probability of the first object having a first attribute based on the second object having a second attribute, the second attribute identified based on the second image crop, the confidence level recalibrated to increase an accuracy of the image classification.

2. The apparatus of claim 1 , wherein the label indicates a product type of the first object and the first attribute of the first object.

3. The apparatus of claim 1 , wherein the confidence recalibrator calculates the probability based on a co-occurrence distribution graph.

4. The apparatus of claim 3 , further including a co-occurrence graph generator to generate a co-occurrence cardinality graph based on labeled data and generate the co-occurrence distribution graph by normalizing the co-occurrence cardinality graph.

5. The apparatus of claim 1 , wherein the trained model is a trained convolutional neural network model.

6. The apparatus of claim 1 , wherein the label is a first label, the confidence level is a first confidence level, the prediction generator to generate a second label corresponding to the second object and a second confidence level associated with the second label, and the confidence recalibrator to recalibrate the first confidence level further based on the second confidence level.

7. The apparatus of claim 1 , wherein the grouping controller selects the second image crop in response to determining that the first object and the second object are positioned on a same shelf.

8. A method for classifying an image, the method comprising:

detecting, by executing instructions with at least one processor, a first image crop from the image, the first image crop corresponding to a first object;

selecting, by executing instructions with the at least one processor, a second image crop corresponding to a second object proximate a location of the first object;

determining, by executing instructions with the at least one processor, (a) a label corresponding to the first object and (b) a confidence level associated with the label based on a trained model; and

recalibrating, by executing instructions with the at least one processor, the confidence level by calculating a probability of the first object having a first attribute based on the second object having a second attribute, the second attribute identified based on the second image crop, the confidence level recalibrated to increase an accuracy of the image classification.

9. The method of claim 8 , wherein the label indicates a product type of the first object and the first attribute of the first object.

10. The method of claim 8 , further including determining, by executing instructions with the at least one processor, the probability based on a co-occurrence distribution graph.

11. The method of claim 10 , further including generating, by executing instructions with the at least one processor, a co-occurrence cardinality graph based on labeled data and generating, by executing instructions with the at least one processor, the co-occurrence distribution graph by normalizing the co-occurrence cardinality graph.

12. The method of claim 8 , wherein the trained model is a trained convolutional neural network model.

13. The method of claim 8 , wherein the label is a first label, the confidence level is a first confidence level, and further including generating, by executing instructions with the at least one processor, a second label corresponding to the second object and a second confidence level associated with the second label, and recalibrating, by executing instructions with the at least one processor, the first confidence level further based on the second confidence level.

14. The method of claim 8 , wherein the selecting the second image crop includes determining that the first object and the second object are positioned on a same shelf.

15. A non-transitory computer readable medium comprising instructions which, when executed, cause at least one processor to:

detect a first image crop from an image, the first image crop corresponding to a first object;

select a second image crop corresponding to a second object proximate a location of the first object;

determine, in response to executing a trained model, (a) a label corresponding to the first object and (b) a confidence level associated with the label; and

recalibrate the confidence level based on calculating a probability of the first object having a first attribute based on the second object having a second attribute, the second attribute identified based on the second image crop, the confidence level recalibrated to increase an accuracy of a classification of the image.

16. The non-transitory computer readable medium of claim 15 , wherein the label indicates a product type of the first object and the first attribute of the first object.

17. The non-transitory computer readable medium of claim 15 , wherein the instructions, when executed, further cause the at least one processor to calculate the probability based on a co-occurrence distribution graph.

18. The non-transitory computer readable medium of claim 17 , wherein the instructions, when executed, further cause the at least one processor to generate a co-occurrence cardinality graph based on labeled data and generate the co-occurrence distribution graph by normalizing the co-occurrence cardinality graph.

19. The non-transitory computer readable medium of claim 15 , wherein the trained model is a trained convolutional neural network model.

20. The non-transitory computer readable medium of claim 15 , wherein the label is a first label, the confidence level is a first confidence level, and wherein the instructions, when executed, further cause the at least one processor to generate a second label corresponding to the second object and a second confidence level associated with the second label, and recalibrate the first confidence level further based on the second confidence level.

21. The non-transitory computer readable medium of claim 20 , wherein the instructions, when executed, further cause the at least one processor to select the second image crop in response to determining that the first object and the second object are positioned on a same shelf.

Assignments (2)
SECURITY INTEREST Recorded Mar 25, 2021
From: NIELSEN CONSUMER LLC; BYZZER INC.
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT AND COLLATERAL AGENT
Reel/Frame 055742/0719 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 4, 2021
From: ALMAZAN, EMILIO; BEASCOECHEA, AITOR ALLER; VELASCO, JAVIER TOVAR
To: NIELSEN CONSUMER LLC
Reel/Frame 055493/0142 →
Cited By (2)
US 12,561,400 US 12,646,347