Generation of jurisdictions lists for input text
Systems and methods are provided that include a processor executing a program to receive input text, divide the input text into sentences, identify one or a plurality of jurisdiction candidates in the sentences from a predetermined taxonomy to generate a jurisdictions list, transform the jurisdictions list using a type recognition neural network to disambiguate jurisdictions in the jurisdictions list, the type recognition neural network being trained on a labeled ground truth dataset containing pairs of geographic names and tax jurisdiction types, and generate and output the jurisdictions list as a jurisdiction prediction list.
1. A computing system, comprising:
a processor and memory of a computing device, the processor being configured to execute a program using portions of memory to:
receive input text;
divide the input text into sentences;
identify one or a plurality of jurisdiction candidates in the sentences from a predetermined taxonomy to generate a jurisdictions list;
transform the jurisdictions list using a type recognition neural network to disambiguate jurisdictions in the jurisdictions list, the type recognition neural network being trained on a labeled ground truth dataset containing pairs of geographic names and tax jurisdiction types; and
generate and output the jurisdictions list as a jurisdiction prediction list.
2. The computing system of claim 1 , wherein the type recognition neural network includes an entity-aware self-attention mechanism.
3. The computing system of claim 2 , wherein the type recognition neural network is a transformer model.
4. The computing system of claim 3 , wherein the transformer model is a LUKE (Language Understanding with Knowledge-based Embeddings) transformer model.
5. The computing system of claim 1 , wherein labels are assigned to contiguous spans of tokens in the sentences using a named entity recognition neural network.
6. The computing system of claim 5 , wherein the labels include labels for organizations and labels for geopolitical entities.
7. The computing system of claim 5 , wherein the labels are filtered out in accordance with a rules-based filter algorithm.
8. The computing system of claim 7 , wherein the rules-based filter algorithm includes a rule that jurisdiction candidates in the labels not recognized as named entities by the named entity recognition neural network be filtered out.
9. The computing system of claim 7 , wherein the rules-based filter algorithm includes a rule requiring that the labels be labels for organizations or labels for geopolitical entities.
10. The computing system of claim 1 , wherein geographic scope is estimated and connections between nearby place names are used to resolve jurisdictions with the same names in the jurisdictions list.
11. A method comprising steps to:
receive input text;
divide the input text into sentences;
identify one or a plurality of jurisdiction candidates in the sentences from a predetermined taxonomy to generate a jurisdictions list;
transform the jurisdictions list using a type recognition neural network to disambiguate jurisdictions in the jurisdictions list, the type recognition neural network being trained on a labeled ground truth dataset containing pairs of geographic names and tax jurisdiction types; and
generate and output the jurisdictions list as a jurisdiction prediction list.
12. The method of claim 11 , wherein the type recognition neural network includes an entity-aware self-attention mechanism.
13. The method of claim 12 , wherein the type recognition neural network is a LUKE (Language Understanding with Knowledge-based Embeddings) transformer model.
14. The method of claim 11 , wherein labels are assigned to contiguous spans of tokens in the sentences using a named entity recognition neural network.
15. The method of claim 14 , wherein the labels include labels for organizations and labels for geopolitical entities.
16. The method of claim 14 , wherein the labels are filtered out in accordance with a rules-based filter algorithm.
17. The method of claim 16 , wherein the rules-based filter algorithm includes a rule that jurisdiction candidates in the labels not recognized as named entities by the named entity recognition neural network be filtered out.
18. The method of claim 16 , wherein the rules-based filter algorithm includes a rule requiring that the labels be labels for organizations or labels for geopolitical entities.
19. The method of claim 11 , wherein geographic scope is estimated and connections between nearby place names are used to resolve jurisdictions with the same names in the jurisdictions list.
20. A computing system for generating a tax jurisdictions list for a tax law article, comprising:
a processor and memory of a computing device, the processor being configured to execute a program using portions of memory to:
receive input text of the tax law article;
identify one or a plurality of tax jurisdiction candidates in sentences of the tax law article from a predetermined tax jurisdictions taxonomy to generate the tax jurisdictions list;
transform the tax jurisdictions list using a transformer model to disambiguate jurisdictions in the tax jurisdictions list, the transformer model being trained on a labeled ground truth dataset containing pairs of geographic names and tax jurisdiction types; and
generate and output the tax jurisdictions list as a jurisdiction prediction list.