Focusing unstructured data and generating focused data determinations from an unstructured data set
Embodiments provide for improvements in generating focused data from an unstructured data set. The focused data generated from the unstructured data set may provide data insight(s) into analysis of the unstructured data set, and/or provide for improved capabilities for a user to efficiently navigate through relevant data portions of such data utilizing a user interface, even when such relevant data portions are not immediately distinguishable without further processing of the unstructured data set. Some embodiments receive an unstructured data set, extract an identified relevant subset utilizing at least one high-level extractor model, extract low-level relevant data from the identified relevant subset utilizing at least one low-level extractor model, generate fraud probability data by applying at least the low-level relevant data and the identified relevant subset to a fraud processing model, and output at least the fraud probability data, identified relevant subset, and/or low-level relevant data, and/or derivations therefrom.
1. A computer-implemented method comprising:
extracting, by one or more processors, an initial keyword set from an identified relevant subset of an unstructured data set, wherein:
(i) the identified relevant subset is generated based at least in part on at least one high-level extractor model, and
(ii) the initial keyword set is extracted based at least in part on a keyword extraction model that generates a keyword relevance score for an initial keyword of the initial keyword set;
identifying, by the one or more processors, an irrelevant keyword based at least in part on a keyword relevance threshold and the keyword relevance score for the initial keyword of the initial keyword set;
generating, by the one or more processors, an updated keyword set by removing the irrelevant keyword from the initial keyword set;
removing, by the one or more processors and from the updated keyword set, an unknown keyword;
generating, by the one or more processors, a filtered keyword set by applying a dictionary filter model to the updated keyword set; and
outputting, by the one or more processors, at least one keyword from the filtered keyword set.
2. The computer-implemented method of claim 1 , wherein outputting the at least one keyword from the filtered keyword set comprises:
causing rendering of the at least one keyword to a user interface.
3. The computer-implemented method of claim 1 , wherein the dictionary filter model is based at least in part on a central truth source.
4. The computer-implemented method of claim 1 , wherein the initial keyword set is extracted utilizing a local interpretable model-agnostic (LIME) model, a Shapley additive explanations (SHAP) model, or an attention-based machine-learning model.
5. The computer-implemented method of claim 1 , wherein the keyword extraction model generates trusted description data associated with the initial keyword of the initial keyword set, and wherein the computer-implemented method further comprises:
calculating the keyword relevance score for the initial keyword based at least in part on a similarity of a data portion associated with the initial keyword of the initial keyword set with the trusted description data.
6. The computer-implemented method of claim 1 , wherein the keyword relevance score does not exceed the keyword relevance threshold.
7. The computer-implemented method of claim 1 , wherein generating the updated keyword set by removing the irrelevant keyword from the initial keyword set comprises:
determining that the irrelevant keyword is not present in a known dictionary.
8. A computing system comprising a processor and memory including program code, the memory and the program code configured to, when executed by the processor, cause the computing system to:
extract an initial keyword set from an identified relevant subset of an unstructured data set, wherein
(i) the identified relevant subset is generated based at least in part on at least one high-level extractor model, and
(ii) the initial keyword set is extracted based at least in part on a keyword extraction model that generates a keyword relevance score for an initial keyword of the initial keyword set;
identify an irrelevant keyword based at least in part on a keyword relevance threshold and the keyword relevance score for the initial keyword of the initial keyword set;
generate an updated keyword set by removing the irrelevant keyword from the initial keyword set;
remove, from the updated keyword set, an unknown keyword;
generate a filtered keyword set by applying a dictionary filter model to the updated keyword set; and
output at least one keyword from the filtered keyword set.
9. The computing system of claim 8 , wherein outputting the at least one keyword from the filtered keyword set comprises:
cause rendering of the at least one keyword to a user interface.
10. The computing system of claim 8 , wherein the dictionary filter model is based at least in part on a central truth source.
11. The computing system of claim 8 , wherein the initial keyword set is extracted utilizing a local interpretable model-agnostic (LIME) model, a Shapley additive explanations (SHAP) model, or an attention-based machine-learning model.
12. The computing system of claim 8 , wherein the keyword extraction model generates trusted description data associated with the initial keyword of the initial keyword set, and wherein the computing system is further caused to:
calculate the keyword relevance score for the initial keyword based at least in part on a similarity of a data portion associated with the initial keyword of the initial keyword set with the trusted description data.
13. The computing system of claim 8 , wherein the keyword relevance score does not exceed the keyword relevance threshold.
14. The computing system of claim 8 , wherein to generate the updated keyword set by removing the irrelevant keyword from the initial keyword set the computing system is further caused to:
determine that the irrelevant keyword is not present in a known dictionary.
15. A computer program product comprising a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium including instructions that, when executed by a computing system, cause the computing system to:
extract an initial keyword set from an identified relevant subset of an unstructured data set, wherein:
(i) the identified relevant subset is generated based at least in part on at least one high-level extractor model, and
(ii) the initial keyword set is extracted based at least in part on a keyword extraction model that generates a keyword relevance score for an initial keyword of the initial keyword set;
identify an irrelevant keyword based at least in part on a keyword relevance threshold and the keyword relevance score for the initial keyword of the initial keyword set;
generate an updated keyword set by removing the irrelevant keyword from the initial keyword set;
remove, from the updated keyword set, an unknown keyword;
generate a filtered keyword set by applying a dictionary filter model to the updated keyword set; and
output at least one keyword from the filtered keyword set.
16. The computer program product of claim 15 , wherein to output the at least one keyword from the filtered keyword set the computing system is caused to:
cause rendering of the at least one keyword to a user interface.
17. The computer program product of claim 15 , wherein the dictionary filter model is based at least in part on a central truth source.
18. The computer program product of claim 15 , wherein the initial keyword set is extracted utilizing a local interpretable model-agnostic (LIME) model, a Shapley additive explanations (SHAP) model, or an attention-based machine-learning model.
19. The computer program product of claim 15 , wherein the keyword extraction model generates trusted description data associated with the initial keyword of the initial keyword set, and wherein the computing system is further caused to:
calculate the keyword relevance score for the initial keyword based at least in part on a similarity of a data portion associated with the initial keyword of the initial keyword set with the trusted description data.
20. The computer program product of claim 15 , wherein to generate the updated keyword set by remove the irrelevant keyword from the initial keyword set the computing system is further caused to:
determine that the at least one irrelevant keyword is not present in a known dictionary.