System and method for automatically tagging documents
System and methods ( 100 ) for automatically tagging electronic documents are disclosed. An input module receives ( 102 ) an electronic document to be tagged. A preprocessing module then preprocesses ( 104 ) the electronic document to be tagged. The preprocessing of the electronic document comprises extracting a text from the electronic document to be tagged, replacing a number or a date in the extracted text with a predetermined symbol, and tokenizing the extracted text with the predetermined symbol into a plurality of tokens. After the preprocessing ( 104 ), a deep learning module determines ( 106 ) a tag for at least one of the plurality of tokens. The determined tag for the at least one token is then output ( 108 ) by an output module.
1. A computer-implemented method for tagging electronic documents, the computer-implemented method comprising:
receiving, by an input module, an electronic document to be tagged;
preprocessing, by a preprocessing module, the electronic document to be tagged, the preprocessing comprising:
extracting a text from the electronic document to be tagged;
replacing at least one of a numerical amount or a date in the extracted text with a predetermined symbol, wherein the predetermined symbol is not used in the extracted text before replacing at least one of the numerical amount or the date in the extracted text, wherein the predetermined symbol includes a special character that is non-numeric and non-alphabetic; and
tokenizing the extracted text with the predetermined symbol into a plurality of tokens without fragmenting the predetermined symbol, wherein keeping the predetermined symbol unfragmented avoids inaccurate tagging associated with the predetermined symbol;
determining, by a deep learning module, a tag for at least one of the plurality of tokens; and
outputting, by an output module, the determined tag for the at least one of the plurality of tokens.
2. The computer-implemented method of claim 1 , wherein tokenizing the extracted text into the plurality of tokens transfers the predetermined symbol into a single token.
3. The computer-implemented method of claim 1 , wherein the extracted text includes at least one additional element to be replaced, wherein the at least one additional element includes at least one of an additional numerical amount or an additional date, and wherein the preprocessing comprises replacing the at least one additional element with the predetermined symbol.
4. The computer-implemented method of claim 1 , wherein the predetermined symbol represents a characteristic of the numerical amount or of the date, wherein the characteristic includes at least one of a shape, a format or a magnitude.
5. The computer-implemented method of claim 1 , wherein the deep learning module comprises an artificial neural network.
6. The computer-implemented method of claim 1 , wherein the deep learning module comprises a transformer-based deep learning model, and wherein the transformer-based deep learning model is based on a Bidirectional Encoder Representations from Transformers (BERT) model.
7. The computer-implemented method of claim 1 , wherein determining a tag for at least one of the plurality of tokens comprises:
transforming, by a neural network encoder, the plurality of tokens into a plurality of numerical vectors in a latent space; and
mapping, by a decoder, the plurality of numerical vectors into tags, the decoder comprising a dense neural network layer with a softmax activation function.
8. The computer-implemented method of claim 1 , wherein the tag is for a token representing the predetermined symbol associated with the numerical amount or the date in the electronic document to be tagged.
9. The computer-implemented method of claim 1 , wherein the electronic document to be tagged is at least one of a document including financial information or a document from a financial domain; and wherein the tag is associated with an eXtensive Business Reporting Language (XBRL).
10. The computer-implemented method of claim 1 , wherein the deep learning module is trained for determining the tag by:
receiving, by the input module, a plurality of electronic documents as a training dataset, the plurality of electronic documents comprising tags associated with text elements in the plurality of electronic documents;
preprocessing, by the preprocessing module, the plurality of electronic documents, wherein each of the plurality of electronic documents is preprocessed by:
extracting a corresponding text from each of the plurality of electronic documents;
replacing an element in the extracted corresponding text with the predetermined symbol; and
tokenizing the extracted corresponding text with the predetermined symbol into a second plurality of tokens, wherein at least some of the second plurality of tokens are associated with one or more tags; and
training, by a training module, the deep learning module with the second plurality of tokens along with the one or more tags.
11. A computer-implemented method for training a deep learning module, the computer-implemented method comprising:
receiving, by an input module, a plurality of electronic documents as training dataset, the plurality of electronic documents comprising tags associated with text elements in the plurality of electronic documents;
preprocessing, by a preprocessing module, the plurality of electronic documents, wherein each of the plurality of electronic documents is preprocessed by:
extracting a text from each of the plurality of electronic documents;
replacing at least one of a numerical amount or a date in the extracted text with a predetermined symbol, wherein the predetermined symbol is not used in the extracted text before replacing at least one of the numerical amount or the date in the extracted text, wherein the predetermined symbol includes a special character that is non-numeric and non-alphabetic; and
tokenizing the extracted text with the predetermined symbol into a plurality of tokens without fragmenting the predetermined symbol, wherein at least some of the plurality of tokens are associated with the tags, and wherein keeping the predetermined symbol unfragmented avoids inaccurate tagging associated with the predetermined symbol; and
training, by the training module, the deep learning module with the plurality of tokens along with the tags.
12. The computer-implemented method of claim 11 , wherein tokenizing the extracted text into the plurality of tokens transfers the predetermined symbol into a single token.
13. The computer-implemented method of claim 11 , wherein the extracted text includes at least one additional element to be replaced, wherein the at least one additional element includes at least one of an additional numerical amount or an additional date, and wherein the preprocessing comprises replacing the at least one additional element with the predetermined symbol.
14. The computer-implemented method of claim 11 , wherein the predetermined symbol represents a characteristic of the numerical amount or of the date, wherein the characteristic includes at least one of a shape, a format or a magnitude.
15. The computer-implemented method of claim 11 , wherein the deep learning module comprises an artificial neural network.
16. The computer-implemented method of claim 11 , wherein the deep learning module comprises a transformer-based deep learning model, and wherein the transformer-based deep learning model is based on a Bidirectional Encoder Representations from Transformers (BERT) model.
17. The computer-implemented method of claim 11 , wherein the plurality of electronic documents includes at least one of a document including financial information or a document from a financial domain; and wherein the tags associated with text elements in the plurality of electronic documents are tags associated with an eXtensive Business Reporting Language (XBRL).
18. A data processing apparatus, comprising:
at least one processor; and
memory storing computer-readable instructions that, when executed by the at least one processor, cause the data processing apparatus to:
receive, by an input module of the data processing apparatus, an electronic document to be tagged;
preprocess, by a preprocessing module of the data processing apparatus, the electronic document to be tagged, the preprocessing comprising:
extracting a text from the electronic document to be tagged;
replacing at least one of a numerical amount or a date in the extracted text with a predetermined symbol, wherein the predetermined symbol is not used in the extracted text before replacing at least one of the numerical amount or the date in the extracted text, wherein the predetermined symbol includes a special character that is non-numeric and non-alphabetic; and
tokenizing the extracted text with the predetermined symbol into a plurality of tokens without fragmenting the predetermined symbol, wherein keeping the predetermined symbol unfragmented avoids inaccurate tagging associated with the predetermined symbol;
determine, by a deep learning module of the data processing apparatus, a tag for at least one of the plurality of tokens; and
output, by an output module of the data processing apparatus, the determined tag for the at least one of the plurality of tokens.