IP Library Granted Patent US 10,235,585
Granted Patent B2
US 10,235,585 · App. 15/258,325 · Granted Mar 19, 2019

Methods and apparatus to determine the dimensions of a region of interest of a target object from an image using target object landmarks

Inventor: Kevin Deng (Safety Harbor, FL)
Assignee: The Nielsen Company (US)
G06K9/2063G06K9/00456G06K9/00463G06K9/00483G06K9/6211G06T7/0085G06K2009/6213G06K2209/01G06T2207/30176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,235,585
App. No.
15/258,325
Granted
Mar 19, 2019
Kind
B2
Abstract

Methods and apparatus to determine the dimensions of a region of interest of a target object and a class of the target object from an image using target object landmarks are disclosed herein. An example method includes identifying a landmark of a target object in an image based on a match between the landmark and a template landmark; classifying a target object based on the identified landmark; projecting dimensions of the template landmark based on a location of the landmark in the image; and determining a region of interest based on the projected dimensions, the region of interest corresponding to text printed on the target object.

Claims (55)

1. A method comprising:

identifying, by executing instructions with a processor, a landmark of a target object in an image based on a match between the landmark and a template landmark;

determining, by executing instructions with the processor, a position and an orientation of the landmark relative to the image;

projecting, by executing instructions with the processor, dimensions of the template landmark based on the position and the orientation of the landmark in the image by:

generating, by executing instructions with the processor, a left side search area, the left side search area being a first region including a first edge of the landmark;

determining, by executing instructions with the processor, a first landmark edge by analyzing first transition points in the left side search area;

generating, by executing instructions with the processor, a right side search area, the right side search area being a second region including a second edge of the landmark; and

determining, by executing instructions with the processor, a second landmark edge by analyzing second transition points in the right side search area; and

defining, by executing instructions with the processor, a region of interest based on the projected dimensions, the region of interest corresponding to text printed on the target object.

2. The method of claim 1 , further including comparing the landmark to a plurality of template landmarks.

3. The method of claim 1 , wherein the template landmark is transformed to correspond to a size and the orientation of the landmark.

4. The method of claim 1 , further including:

determining a first corner of the region of interest based on a first intersection between a third edge of the template landmark and the first landmark edge; and

determining a second corner of the region of interest based on a second intersection between the third edge of the template landmark and the second landmark edge.

5. The method of claim 4 , further including determining a left side edge of the target object and a right side edge of the target object, the left side edge corresponding to the first landmark edge and the right side edge corresponding to the second landmark edge.

6. The method of claim 5 , wherein the left side edge and the right side edge of the target object has a restricted range in direction and area.

7. The method of claim 5 , further including determining text lines within a third region, the third region corresponding to the third edge of the landmark, the left side edge, the right side edge, and a bottom image line corresponding to a bottom of the image.

8. The method of claim 7 , further including:

determining a third corner of the region of interest based on a third intersection between the left side edge and a text line closest to the bottom image line; and

determining a fourth corner of the region of interest based on a fourth intersection between the right side edge and the text line closest to the bottom image line.

9. The method of claim 8 , wherein the first, second, third, and fourth corners of the region of interest improve an accuracy of text recognition techniques.

10. The method of claim 1 , further including determining a class of the target object based on the matched template landmark.

11. The method of claim 1 , wherein the landmark includes a text header.

12. The method of claim 1 , wherein the image includes an image of a receipt.

13. The method of claim 12 , further including determining data based on the printed text in the region of interest.

14. An apparatus comprising:

a landmark detector to identify a landmark of a target object in an image based on a match between the landmark and a template landmark; and

a corner determiner to:

determine a position and an orientation of the landmark relative to the image,

project dimensions of the template landmark based on the position and the orientation of the landmark relative to the image by:

generating a left side search area, the left side search area being a first region including a first edge of the landmark;

determining a first landmark edge by analyzing first transition points in the left side search area;

generating a right side search area, the right side search area being a second region including a second edge of the landmark; and

determining a second landmark edge by analyzing second transition points in the right side search area; and

define a region of interest based on the projected dimensions, the region of interest corresponding to text printed on the target object.

15. The apparatus of claim 14 , wherein the landmark detector is to compare the landmark to a plurality of template landmarks.

16. The apparatus of claim 14 , wherein the corner determiner is to transform the template landmark to match a size and orientation of the landmark.

17. The apparatus of claim 14 , wherein the corner determiner is to:

determine a first corner of the region of interest based on a first intersection between a third edge of the template landmark and the first landmark edge; and

determine a second corner of the region of interest based on a second intersection between the third edge of the template landmark and the second landmark edge.

18. The apparatus of claim 17 , wherein the corner determiner is to determine a left side edge of the target object and a right side edge of the target object, the left side edge corresponding to the first landmark edge and the right side edge corresponding to the second landmark edge.

19. The apparatus of claim 18 , wherein the left side edge and the right side edge of the target object has a restricted range in direction and area.

20. The apparatus of claim 18 , wherein the corner determiner is to determine text lines within a third region, the third region corresponding to the third edge of the template landmark, the left side edge, the right side edge, and a bottom image line corresponding to a bottom of the image.

21. A tangible computer readable storage medium comprising instructions which, when executed, cause a processor to at least:

identify a landmark of a target object in an image based on a match between the landmark and a template landmark;

determine a position and an orientation of the landmark relative to the image;

project dimensions of the template landmark based on the position and the orientation of the landmark relative to the image by:

generating a left side search area, the left side search area being a first region including a first edge of the landmark;

determining a first landmark edge by analyzing first transition points in the left side search area;

generating a right side search area, the right side search area being a second region including a second edge of the landmark; and

determining a second landmark edge by analyzing second transition points in the right side search area; and

define a region of interest based on the projected dimensions, the region of interest corresponding to text printed on the target object.

22. The tangible computer readable storage medium of claim 21 , wherein the instructions, that when executed, further cause the processor to:

determine a first corner of the region of interest based on a first intersection between a third edge of the landmark and the first landmark edge, and

determine a second corner of the region of interest based on a second intersection between the third edge of the landmark and the second landmark edge.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 26, 2016
From: DENG, KEVIN
To: THE NIELSEN COMPANY (US), LLC.
Reel/Frame 039859/0200 →
Continuity (2)
Provisional Application 62320881 · Apr 11, 2016
Related Publication 20170293819A1 · Oct 12, 2017
Cited By (11)
US 12,190,618 US 12,229,741 US 12,229,805 US 12,277,786 US 12,288,405 US 12,315,283 US 12,322,195 US 12,327,425 US 12,548,360 US 12,620,249 US 12,700,251