Systems and methods for developing and verifying image processing standards for mobile deposit
Systems and methods are provided for assessing whether mobile deposit processing engines meet specified standards for mobile deposit of financial documents. A mobile deposit processing engine (MDE) is evaluated to determine if it can perform technical capabilities for improving the quality of and extracting content from an image of a financial document. A verification process then begins, where the MDE performs the image quality enhancements and text extraction steps on sets of images from a test deck. The results of the processing of the test deck are then evaluated by comparing confidence levels with thresholds to determine if each set of images should be accepted or rejected. Further analysis determines whether any of the sets of images were falsely accepted or rejected in error. An overall error rate is then compared with minimum accuracy criteria, and if the criteria are met, the MDE meets the standard for mobile deposit.
1. A method for verifying image processing accuracy of a mobile deposit processing unit for mobile deposit of a financial document, comprising:
receiving sets of test images of financial documents captured by a mobile device;
performing, by one or more processors, an image processing transaction on each set of test images to improve the image quality of at least one aspect of each set of test images and extract at least one category of text from each set of test images, wherein improving the image quality of at least one aspect of each set of test images includes correcting a geometric distortion, cropping and binarizing each set of test images;
determining a confidence level for each category of extracted text;
comparing the confidence level of each category of extracted text with a corresponding threshold value to accept or reject the image processing transaction for each set of test images;
computing a set of false acceptance rates based on the set of acceptances of the image processing transactions on the sets of test images and a set of false rejection rates based on the set of rejections of the image processing transactions on the sets of test images, wherein the set of false rejection rates includes: a false rejection rate computed based on false image quality rejections, a false rejection rate computed based on false payment amount rejections, and a false rejection rate computed based on inaccurate image quality assurance (IQA) rejections;
verifying an accuracy of the mobile deposit processing unit by comparing each of the set of false acceptance rates with a corresponding false acceptance rate threshold and each of the set of false rejection rates with a corresponding false rejection rate threshold; and
determining a status for the mobile deposit processing unit based on the set of false acceptance rates in view of the accuracy of the mobile deposit processing unit.
2. The method of claim 1 , wherein each set of test images include a first image of a front side of a check and a second image of a back side of the check.
3. The method of claim 1 , wherein a statistically significant portion of the sets of test images are good quality images and a statistically significant portion of the sets of test images are poor quality images.
4. The method of claim 1 , wherein the categories of extracted text include at least one of: MICR line, payment amount and endorsement signature.
5. The method of claim 4 , wherein the image processing transaction is accepted if the MICR line and the payment amount were correctly read.
6. The method of claim 5 , wherein the image processing transaction is accepted if the image quality meets a defined image quality threshold.
7. A system for verifying image processing accuracy of a mobile deposit processing unit for mobile deposit of a financial document, the system comprising:
one or more processors; and
a machine-readable medium comprising instructions stored therein, which when executed by the one or more processors, causes the processors to:
receive sets of test images of financial documents captured by a mobile device;
perform an image processing transaction on each set of test images to improve the image quality of at least one aspect of each set of test images and extract at least one category of text from each set of test images; determine a confidence level for each category of extracted text, wherein to improve the image quality of at least one aspect of each set of test images, the one or more processors to correct a geometric distortion, crop and binarize each set of test images;
compare the confidence level of each category of extracted text with a corresponding threshold value to accept or reject the image processing transaction for each set of test images;
compute a set of false acceptance rates based on the set of acceptances of the image processing transactions on the sets of test images and a set of false rejection rates based on the set of rejections of the image processing transactions on the sets of test images, wherein the set of false rejection rates includes: a false rejection rate computed based on false image quality rejections, a false rejection rate computed based on false payment amount rejections, and a false rejection rate computed based on inaccurate image quality assurance (IQA) rejections;
verify an accuracy of the mobile deposit processing unit by comparing each of the set of false acceptance rates with a corresponding false acceptance rate threshold and each of the set of false rejection rates with a corresponding false rejection rate threshold; and
determine a status for the mobile deposit processing unit based on the set of false acceptance rates in view of the accuracy of the mobile deposit processing unit.
8. The system of claim 7 , wherein each set of test images include a first image of a front side of a check and a second image of a back side of the check.
9. The system of claim 7 , wherein a statistically significant portion of the sets of test images are good quality images and a statistically significant portion of the sets of test images are poor quality images.
10. The system of claim 7 , wherein the categories of extracted text include at least one of: MICR line, payment amount and endorsement signature.
11. The system of claim 10 , wherein the image processing transaction is accepted if the MICR line and the payment amount were correctly read.
12. The system of claim 11 , wherein the image processing transaction is accepted if the image quality meets a defined image quality threshold.
13. The method of claim 1 , wherein computing the set of false rejection rates includes:
identifying a subset of the image processing transactions on a subset of the set of test images known to contain only good quality images;
determining a number of rejections on the subset of the set of test images; and
computing a false rejection rate of the subset as the ratio of the number of rejections to a number of images in the subset of the set of test images.
14. The method of claim 1 , wherein computing the set of false acceptance rates includes:
identifying a subset of the image processing transactions on a subset of the set of test images known to contain only poor quality images;
determining a number of acceptances on the subset of the set of test images; and
computing a false acceptance rate of the subset as the ratio of the number of acceptances to a number of images in the subset of the set of test images.
15. The method of claim 1 , wherein the accuracy of the mobile deposit processing unit is verified if each of the set of false acceptance rates is below the corresponding false acceptance rate threshold and each of the set of false rejection rates is below the corresponding false rejection rate threshold.
16. The system of claim 7 , wherein the accuracy of the mobile deposit processing unit is verified if each of the set of false acceptance rates is below the corresponding false acceptance rate threshold and each of the set of false rejection rates is below the corresponding false rejection rate threshold.