Optical receipt processing
Techniques for providing improved optical character recognition (OCR) for receipts are discussed herein. Some embodiments may provide for a system including one or more servers configured to perform receipt image cleanup, logo identification, and text extraction. The image cleanup may include transforming image data of the receipt by using image parameters values that optimize the logo identification, and performing logo identification using a comparison of the image data with training logos associated with merchants. When a merchant is identified, a second image clean up may be performed by using image parameter values optimized for text extraction. A receipt structure may be used to categorize the extracted text. Improved OCR accuracy is also achieved by applying on format rules of the receipt structure to the extracted text.
1. A system, comprising:
one or more servers configured to:
identify a logo of a receipt based on image data defining an image of the receipt;
determine training logo image data defining a training logo associated with a merchant;
determine a logo match score based on a comparison of the logo of the receipt and the training logo associated with the merchant;
determine a logo match score threshold;
determine whether the logo match score satisfies the logo match score threshold; and
in response to determining that the logo match score satisfies the logo match score threshold:
add an image of the logo to the training logo image data associated with the merchant as a new training logo; and
update the logo match score threshold based on a number of training logos associated with the merchant.
2. The system of claim 1 , wherein the one or more servers configured to update the logo match score threshold includes the one or more servers being configured to increase the logo match score threshold when the number of training logos associated with the merchant is increased.
3. The system of claim 1 , wherein the one or more servers configured to determine the logo match score based on the comparison of the logo of the receipt and the training logo associated with the merchant includes the one or more servers being configured to:
programmatically transform the image data based on one or more image parameters; and
perform a point-by-point comparison of the transformed image data with the training logo.
4. The system of claim 1 , wherein the one or more servers are further configured to, in response to determining that the logo match score fails to satisfy the logo match score threshold, determine a second logo match score between the logo of the receipt and a second training logo associated with the merchant.
5. The system of claim 1 , wherein the one or more servers are further configured to, in response to determining that the logo match score fails to satisfy the logo match score threshold, determine a second logo match score between the logo of the receipt and a second training logo associated with a second merchant.
6. The system of claim 1 , wherein the one or more servers are further configured to, in response to determining that the logo match score satisfies the logo match score threshold:
determine the merchant as being associated with the receipt;
determine a receipt structure associated with the merchant; and
perform a text extraction to generate structured receipt data formatted according to the receipt structure.
7. The system of claim 1 , wherein the one or more servers are further configured to:
determine one or more other logo match scores between the logo of the receipt and one or more other training logos;
determine a ranking of the logo match score and the one or more other logo match scores; and
in response to determining that the logo match score has a higher ranking than the one or more other logo match scores, add the image of the logo to the training logo image data associated with the merchant as a new training logo.
8. The system of claim 7 , wherein the one or more servers are further configured to, in response to determining that the logo match score has a higher ranking than the one or more other logo match scores, update the logo match score threshold based on the number of training logos associated with the merchant.
9. The system of claim 1 , wherein the one or more servers configured to identify the logo of the receipt includes the one or more servers being configured to:
programmatically transform the image data based on one or more image parameters; and
identify an edge of the receipt based on the transformed image data.
10. The system of claim 9 , wherein the one or more servers configured to identify the logo of the receipt includes the one or more servers being configured to identify at least one of a color or a shape of the logo from the transformed image data.
11. A method comprising:
identifying, by one or more servers, a logo of a receipt based on image data defining an image of the receipt;
determining, by the one or more servers, training logo image data defining a training logo associated with a merchant;
determining, by the one or more servers, a logo match score based on a comparison of the logo of the receipt and the training logo associated with the merchant;
determining, by the one or more servers, a logo match score threshold;
determining, by the one or more servers, whether the logo match score satisfies the logo match score threshold; and
in response to determining that the logo match score satisfies the logo match score threshold:
adding, by the one or more servers, an image of the logo to the training logo image data associated with the merchant as a new training logo; and
updating, by the one or more servers, the logo match score threshold based on a number of training logos associated with the merchant.
12. The method of claim 11 , wherein updating, by the one or more servers, the logo match score threshold includes the one or more servers being configured to increase the logo match score threshold when the number of training logos associated with the merchant is increased.
13. The method of claim 11 , wherein determining the logo match score based on the comparison of the logo of the receipt and the training logo associated with the merchant includes the one or more servers being configured to:
programmatically transform the image data based on one or more image parameters; and
perform a point-by-point comparison of the transformed image data with the training logo.
14. The method of claim 11 , further comprising in response to determining that the logo match score fails to satisfy the logo match score threshold, determining a second logo match score between the logo of the receipt and a second training logo associated with the merchant.
15. The method of claim 11 , further comprising, in response to determining that the logo match score fails to satisfy the logo match score threshold, determining a second logo match score between the logo of the receipt and a second training logo associated with a second merchant.
16. The method of claim 11 , further comprising in response to determining that the logo match score satisfies the logo match score threshold:
determining the merchant as being associated with the receipt;
determining a receipt structure associated with the merchant; and
performing a text extraction to generate structured receipt data formatted according to the receipt structure.
17. The method of claim 11 , further comprising:
determining one or more other logo match scores between the logo of the receipt and one or more other training logos;
determining a ranking of the logo match score and the one or more other logo match scores; and
in response to determining that the logo match score has a higher ranking than the one or more other logo match scores, adding the image of the logo to the training logo image data associated with the merchant as a new training logo.
18. The method of claim 17 , further comprising in response to determining that the logo match score has a higher ranking than the one or more other logo match scores, updating the logo match score threshold based on the number of training logos associated with the merchant.
19. The method of claim 11 , further comprising programmatically transforming the image data based on one or more image parameters; and
identifying an edge of the receipt based on the transformed image data.
20. The method of claim 19 , further comprising identifying at least one of a color or a shape of the logo from the transformed image data.
21. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions configured to:
identify a logo of a receipt based on image data defining an image of the receipt;
determine training logo image data defining a training logo associated with a merchant;
determine a logo match score based on a comparison of the logo of the receipt and the training logo associated with the merchant;
determine a logo match score threshold;
determine whether the logo match score satisfies the logo match score threshold; and
in response to determining that the logo match score satisfies the logo match score threshold:
add an image of the logo to the training logo image data associated with the merchant as a new training logo; and
update the logo match score threshold based on a number of training logos associated with the merchant.
22. The computer program product of claim 21 , wherein the computer-executable program code instructions comprise program code instructions further configured to increase the logo match score threshold when the number of training logos associated with the merchant is increased.
23. The computer program product of claim 21 , wherein the computer-executable program code instructions comprise program code instructions further configured to:
programmatically transform the image data based on one or more image parameters; and
perform a point-by-point comparison of the transformed image data with the training logo.
24. The computer program product of claim 21 , wherein the computer-executable program code instructions comprise program code instructions further configured to:
in response to determining that the logo match score fails to satisfy the logo match score threshold, determine a second logo match score between the logo of the receipt and a second training logo associated with the merchant.
25. The computer program product of claim 21 , wherein the computer-executable program code instructions comprise program code instructions further configured to:
in response to determining that the logo match score fails to satisfy the logo match score threshold, determine a second logo match score between the logo of the receipt and a second training logo associated with a second merchant.
26. The computer program product of claim 21 , wherein the computer-executable program code instructions comprise program code instructions further configured to:
in response to determining that the logo match score satisfies the logo match score threshold:
determine the merchant as being associated with the receipt;
determine a receipt structure associated with the merchant; and
perform a text extraction to generate structured receipt data formatted according to the receipt structure.
27. The computer program product of claim 21 , wherein the computer-executable program code instructions comprise program code instructions further configured to:
determine one or more other logo match scores between the logo of the receipt and one or more other training logos;
determine a ranking of the logo match score and the one or more other logo match scores; and
in response to determining that the logo match score has a higher ranking than the one or more other logo match scores, add the image of the logo to the training logo image data associated with the merchant as a new training logo.
28. The computer program product of claim 27 , wherein the computer-executable program code instructions comprise program code instructions further configured to:
in response to determining that the logo match score has a higher ranking than the one or more other logo match scores, update the logo match score threshold based on the number of training logos associated with the merchant.
29. The computer program product of claim 21 , wherein the computer-executable program code instructions comprise program code instructions further configured to:
programmatically transform the image data based on one or more image parameters; and
identify an edge of the receipt based on the transformed image data.
30. The computer program product of claim 29 , wherein the computer-executable program code instructions comprise program code instructions further configured to:
identify at least one of a color or a shape of the logo from the transformed image data.