Systems and methods for grouping and enriching data items accessed from one or more databases for presentation in a user interface
Embodiments of the present disclosure relate to a data analysis system for grouping and enriching data items for presentation to an analyst through a user interface. Data items from one or more data sources are combined into memory-efficient clustered data structures, which may be stored as one or more data tables in a database. Analysis and scoring of those clustered data structures can be performed by utilizing various criteria or rules to generate scores, reports, alerts, or conclusions that may aid an analyst in evaluating the clustered data structures. The analysis and scoring may also be added to the clustered data structures which are stored as one or more data tables in a database. The analyst may be prompted to create a dossier format or specification and to additional enrichments to be performed on the raw data items in the clustered data structures. The system may then search, group, or filter the raw data items based on the analyst-defined dossier format, as well as add enrichments to the data. Some examples of enrichments include changing the way the data is displayed, inserting data located in a separate reference table, or ordering data to help construct timelines, histograms, and/or other visualizations based upon the various attributes of the raw data items. The enriched data may be presented to the analyst through a user interface, in the user-defined format or specification in order to allow the analyst to efficiently evaluate the data clusters in the context of, for example, a risky trading investigation.
1. A computer system, comprising:
one or more computer readable storage devices configured to store:
a plurality of computer executable instructions; and
a plurality of raw data items, wherein each raw data item is associated with a respective one or more attributes; and
one or more hardware computer processors in communication with the one or more computer readable storage devices and configured to execute the plurality of computer executable instructions to cause the computer system to:
generate a first interactive user interface configured to allow a user to define a user interface template, wherein the first interactive user interface includes:
a first one or more input areas configured to receive user inputs for defining a format of the user interface template; and
a second one or more input areas configured to receive user input for indicating data enrichments associated with the user interface template, wherein at least one input area of the second one or more input areas is further configured to receive a reference variable link which is associated with a reference table that allows for the automatic joining of up-to-date data from the reference table to data clusters received;
receive a data cluster comprising a group of raw data items, wherein the group of raw data items comprises one or more of the plurality of raw data items and less than all of the plurality of raw data items, and wherein the group of raw data items are related based at least partly on the respective one or more attributes; and
generate a second interactive user interface to display information associated with the data cluster, wherein the second interactive user interface is generated based at least in part on the user interface template including any data enrichments indicated by the user interface template, and wherein the data enrichments are applied to the data cluster via the reference table in conjunction with generating the second interactive user interface so that up-to-date information is included in the second interactive user interface.
2. The computer system of claim 1 , wherein the plurality of computer executable instructions further cause the computer system to:
perform an analysis on the data cluster based on a scoring strategy; and
add the analysis to the data cluster.
3. The computer system of claim 1 , wherein the second interactive user interface comprises a display area displaying at least a portion of the information associated with the data cluster and any associated attributes in a tabular format.
4. The computer system of claim 1 , wherein the second interactive user interface comprises a display area displaying a timeline of at least a portion of the data cluster organized at least in part by a time-based attribute associated with the portion of the data cluster.
5. The computer system of claim 1 , wherein the user interface template comprises a plurality of sections, and wherein the second interactive user interface is displayed according to the plurality of sections of the user interface template.
6. The computer system of claim 1 , wherein the plurality of computer executable instructions further cause the computer system to:
pack the user interface template into a data package; and
unpack the data package to obtain the user interface template prior to generating the second interactive user interface.
7. The computer system of claim 1 , wherein the group of raw data items is stored in one or more data tables in a database.
8. The computer system of claim 7 , wherein generating the second interactive user interface comprises directly accessing the one or more data tables.
9. The computer system of claim 8 , wherein generating the second interactive user interface further comprises querying the raw data items stored in the one or more data tables, based at least in part upon the user interface template and any data enrichments chosen to be applied.
10. The computer system of claim 9 , wherein one of the data enrichments is a versioning operation for which the user input supplies a first key and a second key, and wherein generating the second interactive user interface further comprises grouping and sub-grouping raw data items in the data cluster or the group of raw data items based on the first key and the second key.
11. The computer system of claim 1 , wherein the first one or more input areas includes an input area for customizing one or more user selectable tabs including defining instructions for display of information associated with the data cluster or information associated with data enrichments applied to the data cluster.
12. A computer-implemented method comprising:
generating a first interactive user interface configured to allow a user to define a user interface template, wherein the first interactive user interface includes:
a first one or more input areas configured to receive user inputs for defining a format of the user interface template; and
a second one or more input areas configured to receive user input for indicating data enrichments associated with the user interface template, wherein at least one input area of the second one or more input areas is further configured to receive a reference variable link which is associated with a reference table that allows for the automatic joining of up-to-date data from the reference table to data clusters received;
receiving a data cluster comprising a group of raw data items, wherein the group of raw data items comprises one or more of the plurality of raw data items and less than all of the plurality of raw data items, and wherein the group of raw data items are related based at least partly on one or more attributes; and
generating a second interactive user interface to display information associated with the data cluster, wherein the second interactive user interface is generated based at least in part on the user interface template including any data enrichments indicated by the user interface template, and wherein the data enrichments are applied to the data cluster via the reference table in conjunction with generating the second interactive user interface so that up-to-date information is included in the second interactive user interface.
13. The computer-implemented method of claim 12 , further comprising:
performing an analysis on the data cluster based on a scoring strategy; and
adding the analysis to the data cluster.
14. The computer-implemented method of claim 12 , wherein the second interactive user interface comprises a display area displaying at least a portion of the information associated with the data cluster and any associated attributes in a tabular format.
15. The computer-implemented method of claim 12 , wherein the second interactive user interface comprises a display area displaying a timeline of at least a portion of the data cluster organized at least in part by a time-based attribute associated with the portion of the data cluster.
16. The computer-implemented method of claim 12 , wherein the user interface template comprises a plurality of sections, and wherein the second interactive user interface is displayed according to the plurality of sections of the user interface template.
17. The computer-implemented method of claim 12 , wherein the group of raw data items is stored in one or more data tables in a database.
18. A non-transitory computer-readable medium storing a set of instructions that are executable by one or more processors to cause the one or more processors to:
generate a first interactive user interface configured to allow a user to define a user interface template, wherein the first interactive user interface includes:
a first one or more input areas configured to receive user inputs for defining a format of the user interface template; and
a second one or more input areas configured to receive user input for indicating data enrichments associated with the user interface template, wherein at least one input area of the second one or more input areas is further configured to receive a reference variable link which is associated with a reference table that allows for the automatic joining of up-to-date data from the reference table to data clusters received;
receive a data cluster comprising a group of raw data items, wherein the group of raw data items comprises one or more of the plurality of raw data items and less than all of the plurality of raw data items, and wherein the group of raw data items are related based at least partly on one or more attributes; and
generate a second interactive user interface to display information associated with the data cluster, wherein the second interactive user interface is generated based at least in part on the user interface template including any data enrichments indicated by the user interface template, and wherein the data enrichments are applied to the data cluster via the reference table in conjunction with generating the second interactive user interface so that up-to-date information is included in the second interactive user interface.
19. The non-transitory computer-readable medium of claim 18 , wherein the instructions are further configured to cause the one or more processors to:
perform an analysis on the data cluster based on a scoring strategy; and
add the analysis to the data cluster.
20. The non-transitory computer-readable medium of claim 18 , wherein the second interactive user interface comprises a display area displaying at least a portion of the information associated with the data cluster and any associated attributes in a tabular format.