Photo-based workflow initiation
Systems and methods are provided for generating a resource transmission request to initiate a workflow associated with resource transmission. In particular, the disclosed technology is directed to processing image data corresponding to a physical notice of a request for resource transmission to generate an electronic resource transmission request. The system captures image data of the notice and extracts data from the image data. As an example, the system matches the extracted data against predetermined forms and determines whether the notice is in a known format. In instances where there is no match (such that the notice has an unknown format), the system uses one or more of heatmaps, rules of locating field data, and/or a field data extraction model to assign respective field names with data values in the extracted data. The heatmap includes regions in the image data with a likelihood of data values corresponding to particular field names.
1 . A computer implemented method, comprising:
receiving image data, wherein the image data is representative of a notice of a request for resource transmission in a physical form;
processing the image data to determine a set of regions, wherein each region of the set of regions corresponds to a field name and comprises a subset of pixels representative of field data for the corresponding field name;
processing each determined region of the image data to extract, from the subset of pixels for the region, the field data for the field name associated with the request for resource transmission, wherein the set of regions includes a region for a recipient of the resource transmission, the processing comprising:
matching textual data extracted from image data for a region with a predetermined form in a set of predetermined forms, the matched predetermined form defining a layout for the physical form corresponding to the determined set of regions;
assigning a data value in the textual data to a field name according to the matched predetermined form, wherein the matched predetermined form associates the field name with a region in the image data where the data value is extracted from; and
generating, based at least in part on the assigned data value, the field data;
generating, based on the extracted field data for each of the determined regions, a request for resource transmission to the recipient; and
transmitting the generated resource transmission request, thereby causing transmission of a resource to the recipient based on the extracted field data.
2 . The computer implemented method of claim 1 , wherein the field data includes the field name and the data value assigned to the field name.
3 . The computer implemented method of claim 1 , further comprising:
obtaining a heatmap associated with the field name in the notice of a request for resource transmission; and
assigning, based on the heatmap, a data value in the extracted textual data with the field name, wherein the heatmap indicates a likelihood of the data value extracted from at a region of interest corresponding to the field name.
4 . The computer implemented method of claim 3 , wherein the heatmap includes statistical data based on an aggregate of a set of the textual data corresponding to the field name and a location information of the textual data appearing in the image data.
5 . The computer implemented method of claim 1 , further comprising:
assigning, using a field data extraction model, a data value from a region of the set of regions with a corresponding field name, wherein the field data extraction model, when trained, predicts a likelihood of the data value extracted from the region corresponding to the corresponding field name.
6 . The computer implemented method of claim 5 , wherein the field data extraction model includes a neural network with multiple layers of processing with co-efficient values that have been updated based on training using training data.
7 . The computer implemented method of claim 1 , further comprising:
assigning, using a predefined rule for identifying a data value associated with a field name, a data value from a region of the set of regions with a corresponding field name, wherein the predefined rule indicates a region of the image data where the data value associated with the field name likely appears.
8 . The computer implemented method of claim 1 , wherein the set of regions comprises one or more of:
a region for a sender name,
a region for a sender address,
a region for a receiver name,
a region for a receiver address,
a region for an account number,
a region for a payer name,
a region for a payer address,
a region for a payee name,
a region for a payee address,
a region indicating an amount of resource for transmission, and
a region indicating an amount of financial resources for remittance.
9 . The computer implemented method of claim 1 , wherein the set of regions define a layout of a plurality of different layouts for physical forms.
10 . A system, the system comprising:
a processor; and
a memory storing computer executable instructions that when executed by the processor cause the system to perform a set of operations, the set of operations comprising:
obtaining image data, wherein the image data corresponds to a notice of a request for resource transmission in a physical form;
processing the image data to determine a format from a set of formats, each format defining a different layout for a physical form, the determined format indicating a set of regions for the image data that each correspond to a field name and comprise a subset of pixels representative of field data for the corresponding field name;
processing each region of the image data to extract, from the subset of pixels for the region, the field data for the field name associated with the request for resource transmission, wherein the set of regions includes a region for a recipient of the resource transmission, the processing comprising:
matching textual data extracted from image data for a region with a predetermined form in a set of predetermined forms that each correspond to a format of the set of formats;
assigning a data value in the textual data to a field name according to the matched predetermined form, wherein the matched predetermined form associates the field name with a region in the image data where the data value is extracted from; and
generating, based at least in part on the assigned data value, the field data;
generating, based on the extracted field data for the set of regions, a request for resource transmission to the recipient; and
transmitting the generated resource transmission request, thereby causing transmission of a resource to the recipient based on the extracted field data.
11 . The system of claim 10 , wherein the field data includes a pair of the field name and the data value assigned to the field name.
12 . The system of claim 10 , the set of operations further comprising:
retrieving a heatmap associated with the field name in the notice of a request for resource transmission;
assigning, based on the heatmap, a data value in the extracted textual data with the field name, wherein the heatmap indicates a likelihood of the data value extracted from at a region of interest corresponding to the field name.
13 . The system of claim 10 , the set of operations further comprising:
assigning, using a field data extraction model, a data value from a region of the set of regions with a corresponding field name, wherein the field data extraction model, when trained, predicts a likelihood of the data value extracted from the region corresponding to the corresponding field name.
14 . The system of claim 10 , the set of operation further comprising:
matching, based on fuzzy matching, textual data in a region with a predetermined form in the set of predetermined forms.
15 . The system of claim 10 , wherein the set of regions define a layout of a plurality of different layouts for physical forms.
16 . A computer-implemented method, comprising:
receiving image data, wherein the image data corresponds to a notice of a request for resource transmission in a physical form;
processing the image data to determine a set of regions of the image data, wherein each region of the set of regions corresponds to a field name and comprises a subset of pixels representative of field data for the corresponding field name;
processing each determined region of the image data to extract, from the subset of pixels for the region, the field data corresponding to the field name of and associated with the request for resource transmission, the processing comprising:
matching textual data extracted from image data for a region with a predetermined form in a set of predetermined forms, the matched predetermined form defining a layout for the physical form corresponding to the determined set of regions;
assigning a data value in the textual data to a field name according to the matched predetermined form, wherein the matched predetermined form associates the field name with a region in the image data where the data value is extracted from; and
generating, based at least in part on the assigned data value, the field data;
generating, based on the extracted field data from the received image data, a request instruction to perform the requested resource transmission to the recipient; and
transmitting the generated resource transmission request instruction, thereby causing transmission of a resource to the recipient based on the extracted field data,
wherein the extracted field data includes a recipient of the resource transmission and the field name is part of a set of field names, the set of field names further comprising at least one of:
a sender name,
a sender address,
a receiver name,
a receiver address, or
an amount of a resource for transmission.
17 . The computer-implemented method of claim 16 , further comprising:
assigning, using a field data extraction model, a data value from a region of the set of regions with a corresponding field name, wherein the field data extraction model, when trained, predicts a likelihood of the data value extracted from the region corresponding to the field name.
18 . The computer-implemented method of claim 17 , wherein the field data extraction model includes a neural network with multiple layers of processing with co-efficient values that have been updated based on training using training data.
19 . The computer-implemented method of claim 16 , wherein the set of field names further comprises:
an account number,
a payer name,
a payer address,
a payee name, and
a payee address.
20 . The method of claim 16 , wherein the set of regions define a layout of a plurality of different layouts for physical forms.