IP Library Granted Patent US 11,410,446
Granted Patent B2
US 11,410,446 · App. 16/692,797 · Granted Aug 9, 2022

Methods, systems, apparatus and articles of manufacture for receipt decoding

Inventors: Kannan Shanmuganathan (Chennai, IN); Hussain Masthan (Chennai, IN); Padmanabhan Soundararajan (Tampa, FL); Jose Javier Yebes Torres (Valladolid, ES); Raju Kumar Allam (Chennai, IN)
Assignee: NIELSEN CONSUMER LLC
G06V30/414G06F40/131G06V30/413G06V30/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,410,446
App. No.
16/692,797
Granted
Aug 9, 2022
Kind
B2
Abstract

Methods, apparatus, systems and articles of manufacture are disclosed for receipt decoding. An example apparatus for processing a receipt associated with a user disclosed herein includes an optical character recognition engine to generate bounding boxes, respective ones of the bounding boxes associated with groups of characters detected in the receipt, the bounding boxes including a first bounding box, a second bounding box and a third bounding box, a word connector to connect the first bounding box to the second bounding box based on (1) an adjacency of the first bounding box to the second bounding box and (2) a difference value from a comparison of a location of the first bounding box to a location of the second bounding box, a line connector to form a line of the ones of the bounding boxes by connecting the third bounding box to the second bounding based on a relationship between the first bounding box and the second bounding box, the line of the ones of the bounding boxes indicative of related receipt fields, and a creditor to generate a report based on the line.

Claims (49)

1. An apparatus for processing a receipt associated with a user, the apparatus comprising:

optical character recognition circuitry to generate bounding boxes including a first bounding box, a second bounding box and a third bounding box, respective ones of the bounding boxes associated with groups of characters detected in the receipt, locations of the bounding boxes defined by a set of coordinates;

word connecting circuitry to:

connect the first bounding box to the second bounding box based on (1) an adjacency of the first bounding box to the second bounding box and (2) a difference value from a comparison of a location of the first bounding box to a location of the second bounding box, the comparison including determining whether the location of the first bounding box and the location of the second bounding box satisfy a connection criterion;

in response to determining the first bounding box and the second bounding box satisfy the connection criterion, move the first bounding box and the second bounding box into a list of connected words; and

in response to determining the first bounding box and the second bounding box do not satisfy the connection criterion, move the first bounding box and the second bounding box into a list of non-connected words;

line connecting circuitry to form a line of the bounding boxes by connecting the third bounding box to the second bounding box based on a relationship between the first bounding box and the second bounding box, the line of the bounding boxes indicative of related receipt fields; and

creditor circuitry to generate a report based on the line.

2. The apparatus of claim 1 , wherein the optical character recognition circuitry is to define the location of the first bounding box via a first set of coordinates and the location of the second bounding box via a second set of coordinates.

3. The apparatus of claim 1 , further including range determining circuitry to determine a quantity of the bounding boxes in range of the second bounding box.

4. The apparatus of claim 3 , wherein the line connecting circuitry is to, in response to the range determining circuitry determining the quantity is two:

determine a first association between the first bounding box and the third bounding box and a second association between the third bounding box and the second bounding box; and

connect the second bounding box to the third bounding box based on the first association and the second association.

5. The apparatus of claim 3 , wherein the line connecting circuitry is to, in response to the range determining circuitry determining the quantity is two:

determine a spatial relationship of the first bounding box and the second bounding box; and

connect the second bounding box to the third bounding box based on the spatial relationship.

6. A method for processing a receipt associated with a user, the method comprising:

generating, by executing an instruction with at least one processor, bounding boxes, the bounding boxes including a first bounding box, a second bounding box and a third bounding box, respective ones of the bounding boxes associated with groups of characters detected in the receipt, locations of the bounding boxes defined by a set of coordinates;

connecting, by executing an instruction with the at least one processor, the first bounding box to the second bounding box based on (1) an adjacency of the first bounding box to the second bounding box and (2) a difference value from a comparison of a location of the first bounding box to a location of the second bounding box, the comparison corresponding to satisfaction of a connection criterion;

in response to determining the first bounding box and the second bounding box satisfy the connection criterion, moving the first bounding box and the second bounding box into a list of connected words;

in response to determining the first bounding box and the second bounding box do not satisfy the connection criterion, moving the first bounding box and the second bounding box into a list of non-connected words;

forming, by executing an instruction with the at least one processor, a line of the bounding boxes by connecting the third bounding box to the second bounding based on a relationship between the first bounding box and the second bounding box, the line of the bounding boxes indicative of related receipt fields; and

generating, by executing an instruction with the at least one processor, a report based on the line.

7. The method of claim 6 , wherein the location of the first bounding box is defined by a first set of coordinates and the location of the second bounding box is defined by a second set of coordinates.

8. The method of claim 6 , further including determining a quantity of the bounding boxes in range of the second bounding box.

9. The method of claim 8 , wherein (1) the quantity is two and (2) forming a line of bounding boxes by connecting the third bounding box to the second bounding based on a relationship between the first bounding box and the second bounding box includes:

determining a first association between the first bounding box and the third bounding box and a second association between the third bounding box and the second bounding box; and

connecting the second bounding box to the third bounding box based on the first association and the second association.

10. The method of claim 8 , wherein (1) the quantity is two and (2) forming a line of bounding boxes by connecting the third bounding box to the second bounding box based on a relationship between the first bounding box and the second bounding box includes:

determining a spatial relationship of the first bounding box and the second bounding box; and

connecting the second bounding box to the third bounding box based on the spatial relationship.

11. A non-transitory computer readable medium including instructions, which when executed, cause a processor to at least:

generate bounding boxes, the bounding boxes including a first bounding box, a second bounding box and a third bounding box, respective ones of the bounding boxes associated with groups of characters detected in a receipt, a location of the bounding boxes defined by a set of coordinates;

connect the first bounding box to the second bounding box based on (1) an adjacency of the first bounding box to the second bounding box and (2) a difference value from a comparison of a location of the first bounding box to a location of the second bounding box, the comparison including determining if the first set of coordinates and the second set of coordinates satisfy a connection criterion;

in response to determining the first set of coordinates and the second set of coordinates satisfy the connection criterion, move the first set of coordinate and the second set of coordinates to a list of connected words;

in response to determining the first set of coordinates and the second set of coordinates do not satisfy the connection criterion, moving the first set of coordinate and the second set of coordinates to a list of non-connected words;

form a line of the bounding boxes by connecting the third bounding box to the second bounding based on a relationship between the first bounding box and the second bounding box, the line of the bounding boxes indicative of related receipt fields; and

generate a report based on the line.

12. The non-transitory computer readable medium of claim 11 , wherein the location of the first bounding box is defined by a first set of coordinates and the location of the second bounding box is defined by a second set of coordinates.

13. The non-transitory computer readable medium of claim 11 , wherein the instructions further cause the processor to determine a quantity of the bounding boxes in range of the second bounding box.

14. The non-transitory computer readable medium of claim 13 , wherein the quantity is two and the instructions further cause the processor to

determine a spatial relationship of the first bounding box and the second bounding box; and

connect the second bounding box to the third bounding box based on the spatial relationship.

15. The apparatus of claim 1 , further including bounding box correcting circuitry to:

determine if one of the bounding boxes is missing a coordinate of the set of coordinates; and

in response to determining one of the bounding boxes is missing the coordinate, assign a default value to the coordinate.

16. The method of claim 6 , further including correcting the respective bounding boxes by:

determining, by executing an instruction with the at least one processor, if one of the bounding boxes is missing a coordinate of the set of coordinates; and

in response to determining one of the bounding boxes is missing the coordinate, assigning, by executing an instruction with the at least one processor, a default value to the coordinate.

Assignments (8)
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
SECURITY INTEREST Recorded Mar 25, 2021
From: NIELSEN CONSUMER LLC; BYZZER INC.
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT AND COLLATERAL AGENT
Reel/Frame 055742/0719 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Mar 10, 2021
From: CITIBANK, N.A.
To: NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN CONSUMER LLC
Reel/Frame 055557/0949 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 17, 2021
From: THE NIELSEN COMPANY (US), LLC
To: NIELSEN CONSUMER LLC
Reel/Frame 055325/0353 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 1, 2020
From: MASTHAN, HUSSAIN; ALLAM, RAJU KUMAR; SHANMUGANATHAN, KANNAN; YEBES TORRES, JOSE JAVIER; SOUNDARARAJAN, PADMANABHAN
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 053663/0424 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →