Condensing a document for enhanced analysis and processing

1 . A computer-implemented method for document processing, comprising:

defining an ontology for a first document, wherein defining the ontology further comprises creating a document fingerprint for the first document, wherein the document fingerprint is a hash of character and image data within the first document;

determining a document type for the first document, based on the ontology associated with the first document, wherein the first document includes a plurality of pages;

training a machine learning process that includes a neural network and a natural language processing (NLP) module;

extracting a plurality of key-value pairs (KVPs) from the first document based on the determined document type, wherein the extracting is performed using the trained machine learning process that includes the neural network and the NLP module;

identifying an excerpt of the first document that contains a KVP of the plurality of KVPs;

creating a condensed document, wherein the condensed document includes the identified excerpt of the first document, and wherein the condensed document excludes at least one other excerpt of the first document that is not identified as containing a KVP of the plurality of KVPs;

performing indexing on the condensed document by:

performing preprocessing on the condensed document, wherein the preprocessing includes tokenization, lowercasing, and removal of stop words;

extracting individual terms from the preprocessed condensed document;

filtering out a portion of the individual terms to generate a subset of terms; and

characterizing a frequency of each of the subset of terms; and

providing the condensed document to a downstream application, and wherein, during a runtime operation, the downstream application is configured to utilize input from the condensed document.

2 . The method of claim 1 , wherein defining the ontology further comprises initializing a mandatory flag for each KVP of the plurality of KVPs.

3 . The method of claim 2 , further comprising performing validation on each KVP of the plurality of KVPs, based on a setting of the mandatory flag.

4 . The method of claim 3 , wherein the performing the validation comprises:

in response to a first KVP of the plurality of KVPs not appearing in the condensed document:

generating an alert indicating that the first KVP does not appear in the condensed document.

5 . The method of claim 3 , wherein the performing the validation comprises:

in response to a first KVP of the plurality of KVPs not appearing in the condensed document:

retraining the machine learning process to include the first KVP of the plurality of KVPs in the condensed document.

6 . The method of claim 1 , further comprising:

identifying a supplemental excerpt of the first document based on the natural language processing module, the supplemental excerpt containing metadata associated with the KVP; and

including the supplemental excerpt in the condensed document.

7 . The method of claim 1 , wherein the extracting is performed using regular expressions.

8 . The method of claim 1 , wherein the KVP is one of a plurality of KVPs stored in a JSON file, wherein the excerpt is a page, and wherein page location data associated with the KVP is stored in the JSON file.

9 . The method of claim 8 , wherein the page location data includes a page number, and Cartesian coordinates of the KVP.

10 . The method of claim 9 , wherein the page location data includes a height of the KVP and a width of the KVP.

11 . The method of claim 1 , further comprising:

receiving, from a client device, search terms related to the first document, wherein a radio button identifying the condensed document is selected on a user interface of the client device; and

executing a search of the search terms on the indexed condensed document based on the radio button identifying the condensed document being selected.

12 . An electronic computation device comprising:

a processor;

a memory coupled to the processor, the memory containing instructions, that when executed by the processor, cause the electronic computation device to:

define an ontology for a first document, wherein defining the ontology further comprises creating a document fingerprint for the first document, wherein the document fingerprint is a hash of character and image data within the first document;

determine a document type for the first document, based on the ontology associated with the first document, wherein the first document includes a plurality of pages;

train a machine learning process that includes a neural network and a natural language processing (NLP) module;

extract a plurality of key-value pairs (KVPs) from the first document based on the determined document type, wherein the extracting is performed using the trained machine learning process that includes the neural network and the NLP module;

identify an excerpt of the first document that contains a KVP of the plurality of KVPs;

create a condensed document, wherein the condensed document includes the identified excerpt of the first document, and wherein the condensed document excludes at least one other excerpt of the first document that is not identified as containing a KVP of the plurality of KVPs; and

perform indexing on the condensed document by:

performing preprocessing on the condensed document, wherein the preprocessing includes tokenization, lowercasing, and removal of stop words;

extracting individual terms from the preprocessed condensed document;

filtering out a portion of the individual terms to generate a subset of terms; and

characterizing a frequency of each of the subset of terms; and

provide the condensed document to a downstream application, and wherein, during a runtime operation, the downstream application is configured to utilize input from the condensed document.

13 . The electronic computation device of claim 12 , wherein the KVP is one of a plurality of KVPs stored in a JSON file, wherein the excerpt is a page, wherein page location data associated with the KVP is stored in the JSON file, wherein the page location data includes a page number, Cartesian coordinates of the KVP, a height of the KVP, and a width of the KVP.

14 . The electronic computation device of claim 12 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to initialize a mandatory flag for each KVP of the plurality of KVPs.

15 . The electronic computation device of claim 14 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to perform validation on each KVP of the plurality of KVPs, based on a setting of the mandatory flag.

16 . A computer program product for an electronic computation device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the electronic computation device to:

define an ontology for a first document, wherein defining the ontology further comprises creating a document fingerprint for the first document, wherein the document fingerprint is a hash of character and image data within the first document;

determine a document type for the first document, based on the ontology associated with the first document, wherein the first document includes a plurality of pages;

train a machine learning process that includes a neural network and a natural language processing (NLP) module;

extract a plurality of key-value pairs (KVPs) from the first document based on the determined document type, wherein the extracting is performed using the trained machine learning process that includes the neural network and the NLP module;

identify an excerpt of the first document that contains a KVP of the plurality of KVPs;

create a condensed document, wherein the condensed document includes the identified excerpt of the first document, and wherein the condensed document excludes at least one other excerpt of the first document that is not identified as containing a KVP of the plurality of KVPs; and

perform indexing on the condensed document by:

performing preprocessing on the condensed document, wherein the preprocessing includes tokenization, lowercasing, and removal of stop words;

extracting individual terms from the preprocessed condensed document;

filtering out a portion of the individual terms to generate a subset of terms; and

characterizing a frequency of each of the subset of terms; and

provide the condensed document to a downstream application, and wherein, during a runtime operation, the downstream application is configured to utilize input from the condensed document.

17 . The computer program product of claim 16 , wherein the computer readable storage medium further comprises program instructions, that when executed by the processor, cause the electronic computation device to initialize a mandatory flag for each KVP of the plurality of KVPs.

18 . The computer program product of claim 17 , wherein the computer readable storage medium further comprises program instructions, that when executed by the processor, cause the electronic computation device to perform validation on each KVP of the plurality of KVPs, based on a setting of the mandatory flag.

19 . The computer program product of claim 16 , wherein the KVP is one of a plurality of KVPs stored in a JSON file, wherein the excerpt is a page, wherein page location data associated with the KVP is stored in the JSON file, wherein the page location data includes a page number, Cartesian coordinates of the KVP, a height of the KVP, and a width of the KVP.