Machine-learning models for image processing
Presented herein are systems and methods for the employment of machine learning models for image processing as may be performed by computing devices associated with an end user. A method may include obtaining video data comprising a plurality of frames including a document of a document type. The method may include executing an object recognition engine of a machine-learning architecture using image data of the plurality of frames, the object recognition engine trained to detect edges of documents. The method may include identifying, based on the edge detection, a plurality of boundaries for the document. The method may include validating, based on the plurality of boundaries, the document as the document type. The method may include transmitting via one or more networks, to a computer remote from the computing device, responsive to the validation of the type of document, the image data for the plurality of frames depicting the document.
1. A method for client-side validation of document-imagery for remote processing, the method comprising:
obtaining, by a camera of a mobile client device associated with an end-user, video data comprising a plurality of frames including a document of a document type of the document;
executing, by the mobile client device, an object recognition engine of a machine-learning architecture to extract a first set of document features from image data of the plurality of frames captured by the camera of the mobile client device;
executing, by the mobile client device, the object recognition engine on the first set of document features to detect the document type of the document based upon the first set of document features, the object recognition engine trained to detect the document type of the document using a set of document features and corresponding training labels indicating the document type of the document having the set of document features;
executing, by the mobile client device, the object recognition engine to extract a second set of document features from the image data, the second set of document features are extracted based upon the document type of the document detected using the first set of document features;
generating, by the mobile client device, a document validation score indicating a likelihood that the document is a valid document based upon the second set of document features extracted from the image data; and
upon validating the document based on determining that the document validation score satisfies a document validation threshold,
generating, by the mobile client device, a packaged image extracted from the image data of at least one frame of the plurality of frames of the video data captured by the mobile client device; and
generating, by the mobile client device, an operation instruction for a backend server, the operation instruction to include the packaged image and mobile client device metadata identifying the mobile client device.
2. The method of claim 1 , wherein validating the document includes identifying, by the mobile client device, a dimension similarity between the document and the document type of the document, based upon comparing a plurality of boundaries corresponding to a plurality of edges of the document in the first set of document features against a predefined dimension having a plurality of expected boundaries for the document type of the document.
3. The method of claim 2 , wherein the identifying the dimension similarity includes matching the plurality of boundaries to the predefined dimension having the plurality of expected boundaries for a plurality of document issuers, wherein each of the plurality of expected boundaries correspond to a document issuer of the plurality of document issuers.
4. The method of claim 1 , further comprising generating, by the mobile client device, an indication of validation for presenting via a user interface of the mobile client device.
5. The method of claim 1 , further comprising generating, by the mobile client device, an indication that the document validation score exceeds a warning threshold for presenting via a user interface of the mobile client device.
6. The method of claim 1 , wherein validating the document includes:
comparing, by the mobile client device, the document validation score against at least one of a second validation threshold corresponding to non-validation and a warning threshold corresponding to an alert trigger; and
generating, by the mobile client device, an output indicator for display at a user interface, based upon comparing the document validation score against the at least one of the second validation threshold or the warning threshold.
7. The method of claim 1 , wherein validating the document includes identifying, by the mobile client device, at least one of a first set of validation criteria corresponding to clerical or imaging errors or a second set of validation criteria corresponding to digital or mechanical manipulation of the document.
8. The method of claim 1 , wherein the machine-learning architecture comprises a classification model configured to classify the document the document type of the document.
9. The method of claim 1 , wherein the machine-learning architecture includes an edge-detection model configured to detect one or more edges of a rectangle bounding the document for validating the document.
10. The method of claim 1 , further comprising generating, by the mobile client device, a spatial transform for the document responsive to detecting a plurality of edges of the document of the first set of document features.
11. The method of claim 1 , wherein validating the document includes:
identifying, by the mobile client device, a subset of the video data including the document; and
comparing, by the mobile client device, the subset against an occupancy threshold.
12. The method of claim 1 , wherein the image data transmitted to the backend server remote from the mobile client device is transmitted in a video feed comprising multiple of the plurality of frames of the video data.
13. A system for client-side validation of document-imagery for remote processing, the system comprising:
a mobile client device associated with an end-user comprising at least one processor and a camera, configured to:
obtain, by the camera, video data comprising a plurality of frames including a document of a document type of the document;
execute an object recognition engine of a machine-learning architecture to extract a first set of document features from image data of the plurality of frames captured by the camera of the mobile client device;
execute the object recognition engine on the first set of document features to detect the document type of the document based upon the first set of document features, the object recognition engine trained to detect the document type using a set of document features and corresponding training labels indicating the document type of the document having the set of document features;
execute the object recognition engine to extract a second set of document features from the image data, the second set of document features are extracted based upon the document type of the document detected using the first set of document features;
generate a document validation score indicating a likelihood that the document is a valid document based upon the second set of document features extracted from the image data; and
upon the mobile client device validating the document based on determining that the document validation score satisfies a document validation threshold,
generate a packaged image extracted from the image data of at least one frame of the plurality of frames of the video data captured by the mobile client device; and
generate an operation instruction for a backend server, the operation instruction to include the packaged image and mobile client device metadata identifying the mobile client device.
14. The system of claim 13 , wherein when validating the document the mobile client device is configured to identify a dimension similarity between the document and the document type of the document, based upon comparing a plurality of boundaries corresponding to a plurality of edges of the document in the first set of document features against a predefined dimension having a plurality of expected boundaries for the document type of the document.
15. The system of claim 13 , wherein the mobile client device is further configured to generate an indication of validation for presenting via a user interface of the mobile client device.
16. The system of claim 13 , wherein the mobile client device is further configured to generate an indication that the document validation score exceeds a warning threshold for presenting via a user interface of the mobile client device.
17. The system of claim 13 , wherein when validating the document the mobile client device is configured to:
compare the document validation score against at least one of a second validation threshold corresponding to non-validation and a warning threshold corresponding to an alert trigger; and
generate an output indicator for display at a user interface, based upon comparing the document validation score against the at least one of the second validation threshold or the warning threshold.
18. The system of claim 13 , wherein when validating the document the mobile client device is configured to identify at least one of a first set of validation criteria corresponding to clerical or imaging errors or a second set of validation criteria corresponding to digital or mechanical manipulation of the document.
19. The system of claim 13 , wherein the mobile client device is further configured to generate a spatial transform for the document responsive to detecting a plurality of edges of the document of the first set of document features.
20. The system of claim 13 , wherein the mobile client device is further configured to transmit the operation instruction to the backend server via one or more networks, the operation instruction including the packaged image and device metadata identifying the mobile client device.