IP Library Granted Patent US 10,860,884
Granted Patent B2
US 10,860,884 · App. 16/282,060 · Granted Dec 8, 2020

Methods and apparatus to determine the dimensions of a region of interest of a target object from an image using target object landmarks

Inventor: Kevin Deng (Safety Harbor, FL)
Assignee: The Nielsen Company (US), LLC
G06K9/6211G06K9/00456G06K9/00463G06K9/00483G06K9/2063G06K9/3233G06K2009/6213G06K2209/01G06T2207/30176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,860,884
App. No.
16/282,060
Granted
Dec 8, 2020
Kind
B2
Abstract

Methods and apparatus to determine the dimensions of a region of interest of a target object and a class of the target object from an image using target object landmarks are disclosed herein. An example method includes identifying a landmark of a target object in an image based on a match between the landmark and a template landmark; classifying a target object based on the identified landmark; projecting dimensions of the template landmark based on a location of the landmark in the image; and determining a region of interest based on the projected dimensions, the region of interest corresponding to text printed on the target object.

Claims (55)

1. An apparatus comprising:

a landmark detector implemented by at least one processor to identify a landmark in an image based on a match between the landmark and a template landmark; and

a corner determiner implemented by at least one processor to:

generate a first side search area based on a first template edge of the template landmark, the first side search area being a first region including a first landmark edge of the landmark in the image;

identify the first landmark edge by analyzing first transition points in the first side search area;

generate a second side search area based on a second template edge of the template landmark, the second side search area being a second region including a second landmark edge of the landmark; and

identify the second landmark edge by analyzing second transition points in the second side search area; and

identify a region of interest of the image based on the first landmark edge and the second landmark edge, the region of interest corresponding to text printed in the image.

2. The apparatus of claim 1 , wherein the landmark detector is to match the landmark to the template landmark by comparing the landmark to a plurality of template landmarks.

3. The apparatus of claim 1 , wherein the corner determiner is to transform a template landmark size and a template landmark orientation of the template landmark to match a landmark size and a landmark orientation of the landmark.

4. The apparatus of claim 1 , wherein the corner determiner is to:

determine a first corner of the region of interest based on a first intersection between a third template edge of the template landmark and the first landmark edge; and

determine a second corner of the region of interest based on a second intersection between the third template edge of the template landmark and the second landmark edge; and

determine a first side edge of the region of interest and a second side edge of the region of interest, the first side edge corresponding to the first landmark edge and the second side edge corresponding to the second landmark edge.

5. The apparatus of claim 4 , wherein the first side search area and the second side search area are limited in direction and area.

6. The apparatus of claim 4 , wherein the corner determiner is to determine text lines within a third region, the third region corresponding to the third template edge of the landmark, the first side edge, the second side edge, and a bottom image line corresponding to a bottom of the image.

7. The apparatus of claim 6 , wherein the corner determiner is to:

determine a third corner of the region of interest based on a third intersection between the first side edge and a text line closest to the bottom image line; and

determine a fourth corner of the region of interest based on a fourth intersection between the second side edge and the text line closest to the bottom image line.

8. The apparatus of claim 7 , wherein the first, second, third, and fourth corners of the region of interest improve an accuracy of text recognition.

9. The apparatus of claim 1 , wherein the landmark detector is to determine a class of an object included in the image based on the matched template landmark, the region of interest included in the object, the class including at least one of a company or a store.

10. An apparatus comprising:

first means for identifying a landmark in an image based on a match between the landmark and a template landmark; and

second means for:

generating a first side search area based on a first template edge of the template landmark, the first side search area being a first region including a first landmark edge of the landmark in the image;

identifying the first landmark edge by analyzing first transition points in the first side search area;

generating a second side search area based on a second template edge of the template landmark, the second side search area being a second region including a second landmark edge of the landmark; and

identifying the second landmark edge by analyzing second transition points in the second side search area; and

identifying a region of interest of the image based on the first landmark edge and the second landmark edge, the region of interest corresponding to text printed in the image.

11. The apparatus of claim 10 , wherein the first means is to match the landmark to the template landmark by comparing the landmark to a plurality of template landmarks.

12. The apparatus of claim 10 , wherein the second means is to transform a template landmark size and a template landmark orientation of the template landmark to match a landmark size and a landmark orientation of the landmark.

13. The apparatus of claim 10 , wherein the second means is to:

determine a first corner of the region of interest based on a first intersection between a third template edge of the template landmark and the first landmark edge; and

determine a second corner of the region of interest based on a second intersection between the third template edge of the template landmark and the second landmark edge; and

determine a first side edge of the region of interest and a second side edge of the region of interest, the first side edge corresponding to the first landmark edge and the second side edge corresponding to the second landmark edge.

14. The apparatus of claim 13 , wherein the second means is to determine text lines within a third region, the third region corresponding to the third template edge of the landmark, the first side edge, the second side edge, and a bottom image line corresponding to a bottom of the image.

15. The apparatus of claim 14 , wherein the second means is to:

determine a third corner of the region of interest based on a third intersection between the first side edge and a text line closest to the bottom image line; and

determine a fourth corner of the region of interest based on a fourth intersection between the second side edge and the text line closest to the bottom image line.

16. The apparatus of claim 10 , wherein the first means is to determine a class of an object included in the image based on the matched template landmark, the object including the region of interest, the class including at least one of a company or store.

17. A tangible computer readable storage medium comprising instructions which, when executed, cause a processor to at least:

identify a landmark in an image based on a match between the landmark and a template landmark;

determine a first landmark edge by analyzing transition points in a first side search area corresponding to a first edge of the template landmark;

determine a second landmark edge by analyzing second transition points in a second side search area corresponding to a second edge of the template landmark;

determine a first side edge corresponding to the first landmark edge and a second side edge corresponding to the second landmark edge;

identify text lines within a region defined by the first side edge and the second side edge; and

identify a region of interest based on the text lines, the region of interest corresponding to text printed in the image.

18. The tangible computer readable storage medium of claim 17 , wherein the instructions cause the processor to:

determine a first corner of the region of interest based on a first intersection between a third edge of the template landmark and the first landmark edge; and

determine a second corner of the region of interest based on a second intersection between the third edge of the template landmark and the second landmark edge.

19. The tangible computer readable storage medium of claim 18 , wherein the first edge is an edge of an object included in the image, the object including the region of interest, and wherein the instructions cause the processor to determine the first side edge of the object by:

extending the first side edge to an image line corresponding to an end of the image;

generating a plurality of lines between the first corner and a bottom image line within a threshold range of intersection; and

selecting the first side edge based on transition point counts for the plurality of lines.

20. The tangible computer readable storage medium of claim 18 , wherein the instructions cause the processor to identify the text lines based on dark-to-bright transition points.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 8, 2019
From: DENG, KEVIN
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 048824/0580 →
Continuity (3)
Continuation 15258325 · Sep 7, 2016
Provisional Application 62320881 · Apr 11, 2016
Related Publication 20190188527A1 · Jun 20, 2019