Automated data extraction and document generation
A computer-implemented method of generating electronic documents is described. The method comprises receiving a plurality of scanned documents for a plurality of vehicles; providing the plurality of scanned documents to a neural network model that outputs respective class identifiers of the plurality of scanned documents; for each scanned document of the plurality of scanned documents, extracting data from the scanned document according to a corresponding class identifier, and associating the scanned document and the extracted data with an identified vehicle of the plurality of vehicles, wherein the identified vehicle is identified by the extracted data; and generating an electronic document for the identified vehicle using the extracted data.
1. A computer-implemented method of generating electronic documents, the method comprising:
receiving a plurality of scanned documents for a plurality of vehicles;
providing the plurality of scanned documents to a neural network model that outputs respective class identifiers of the plurality of scanned documents;
for each scanned document of the plurality of scanned documents,
extracting data from the scanned document according to a corresponding class identifier, and
associating the scanned document and the extracted data with an identified vehicle of the plurality of vehicles, wherein the identified vehicle is identified by the extracted data; and
generating an electronic document for the identified vehicle using the extracted data.
2. The computer-implemented method of claim 1 , further comprising:
modifying a first plurality of training documents having a first document type to include labels of data to be extracted;
training the neural network model to classify scanned documents having the first document type according to a first class identifier using the first plurality of training documents.
3. The computer-implemented method of claim 2 , further comprising:
pre-processing the first plurality of training documents, including converting the first plurality of training documents into images and using one or more image augmentation processes.
4. The computer-implemented method of claim 3 , wherein the one or more image augmentation processes include a realignment process that corrects an angular orientation of an image.
5. The computer-implemented method of claim 3 , wherein the one or more image augmentation processes include a noise reduction process that corrects noise within an image.
6. The computer-implemented method of claim 2 , further comprising:
storing at least some of the plurality of scanned documents that are unclassifiable by the neural network model as a second plurality of training documents;
retraining the neural network model to classify scanned documents using the first plurality of training documents and the second plurality of training documents.
7. The computer-implemented method of claim 6 , further comprising modifying the second plurality of training documents to include labels of data to be extracted before retraining the neural network model.
8. The computer-implemented method of claim 6 , further comprising flagging the unclassifiable documents to be manually processed before the retraining of the neural network model.
9. The computer-implemented method of claim 1 , further comprising:
verifying the extracted data by comparing the extracted data with existing data associated with the identified vehicle in a data store;
storing the extracted data in the data store when verification is successful;
flagging the extracted data to be manually processed when verification is unsuccessful.
10. The computer-implemented method of claim 9 , wherein verifying the extracted data includes performing a Soundex match between the extracted data and the existing data.
11. The computer-implemented method of claim 10 , further including automatically correcting the extracted data using the existing data when the Soundex match is successful.
12. A document intake system, comprising:
a processor;
a neural network model; and
a memory storing computer-executable instructions that when executed by the processor cause the system to:
receive a plurality of scanned documents for a plurality of vehicles;
provide the plurality of scanned documents to a neural network model that outputs respective class identifiers of the plurality of scanned documents;
for each scanned document of the plurality of scanned documents,
extract data from the scanned document according to a corresponding class identifier, and
associate the scanned document and the extracted data with an identified vehicle of the plurality of vehicles, wherein the identified vehicle is identified by the extracted data; and
generate an electronic document for the identified vehicle using the extracted data.
13. The document intake system of claim 12 , wherein the computer-executable instructions cause the system to:
modify a first plurality of training documents having a first document type to include labels of data to be extracted;
train the neural network model to classify scanned documents having the first document type according to a first class identifier using the first plurality of training documents.
14. The document intake system of claim 13 , wherein the computer-executable instructions cause the system to: pre-process the first plurality of training documents, including converting the first plurality of training documents into images and using one or more image augmentation processes.
15. The document intake system of claim 14 , wherein the one or more image augmentation processes include a realignment process that corrects an angular orientation of an image.
16. The document intake system of claim 14 , wherein the one or more image augmentation processes include a noise reduction process that corrects noise within an image.
17. The document intake system of claim 13 , wherein the computer-executable instructions cause the system to: store at least some of the plurality of scanned documents that are unclassifiable by the neural network model as a second plurality of training documents; retrain the neural network model to classify scanned documents using the first plurality of training documents and the second plurality of training documents.
18. The document intake system of claim 17 , wherein the computer-executable instructions cause the system to: modify the second plurality of training documents to include labels of data to be extracted before retraining the neural network model.
19. The document intake system of claim 17 , wherein the computer-executable instructions cause the system to: flag the unclassifiable documents to be manually processed before the retraining of the neural network model.
20. The document intake system of claim 12 , wherein the computer-executable instructions cause the system to: verify the extracted data by comparing the extracted data with existing data associated with the identified vehicle in a data store: store the extracted data in the data store when verification is successful: flag the extracted data to be manually processed when verification is unsuccessful.