Automatic data domain identification
An apparatus is disclosed which includes at least one processing device comprising a processor coupled to a memory. The at least one processing device, when executing program code, is configured to: extract one or more entities identified in a plurality of data artifacts based at least in part on one or more datasets, extract one or more entities identified in a plurality of code artifacts based at least in part on the one or more datasets, extract one or more entities identified in a plurality of user interface artifacts based at least in part on the one or more datasets, generate a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities, and perform one or more of a lexical analysis and a semantic analysis on the set of dependency graphs to identify a data domain of the one or more datasets.
1. An apparatus, comprising:
at least one processing device comprising a processor coupled to a memory, the at least one processing device, when executing program code, is configured to:
extract one or more entities identified in a plurality of data artifacts based at least in part on one or more datasets;
extract one or more entities identified in a plurality of code artifacts based at least in part on the one or more datasets;
extract one or more entities identified in a plurality of user interface artifacts based at least in part on the one or more datasets;
generate a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities; and
perform one or more of a lexical analysis and a semantic analysis on the set of dependency graphs to identify a data domain of the one or more datasets.
2. The apparatus of claim 1 , wherein the plurality of data artifacts comprises one or more of (a) one or more schemas with their table names and associated column names, (b) index, trigger, and stored procedures associated with the schemas, (c) relationships between the different tables and databases, (d) table data, and (e) documentation, logs, performance and operational profile of datasets.
3. The apparatus of claim 1 , wherein the plurality of code artifacts comprises source code and associated libraries.
4. The apparatus of claim 1 , wherein the plurality of user interface artifacts comprises one or more of (a) user interface screens with natural language text, (b) user interface form objects and formatting, and (c) user interface modalities.
5. The apparatus of claim 1 , wherein generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities comprises generating a first dependency graph of the set of dependency graphs based at least in part on one or more relationships between the extracted one or more entities of the data artifacts and the extracted one or more entities of the code artifacts.
6. The apparatus of claim 5 , wherein generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities further comprises generating a second dependency graph of the set of dependency graphs based at least in part on one or more relationships between the extracted one or more entities of the data artifacts and the extracted one or more entities of the user interface artifacts.
7. The apparatus of claim 6 , wherein generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities further comprises generating a third dependency graph of the set of dependency graphs based at least in part on one or more relationships between the extracted one or more entities of the code artifacts and the extracted one or more entities of the user interface artifacts.
8. The apparatus of claim 1 , wherein the at least one processing device, when executing program code, is further configured to:
retrieve the data artifacts from at least one of a database and a file system; and
apply one of a data definition language operation and a data manipulation language operation to identify the one or more entities in each data artifact and to determine one or more relationships between the one or more entities.
9. A computer-implemented method, comprising:
extracting one or more entities identified in a plurality of data artifacts based at least in part on one or more datasets;
extracting one or more entities identified in a plurality of code artifacts based at least in part on the one or more datasets;
extracting one or more entities identified in a plurality of user interface artifacts based at least in part on the one or more datasets;
generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities; and
performing one or more of a lexical analysis and a semantic analysis on the set of dependency graphs to identify a data domain of the one or more datasets;
wherein the method is carried out by at least one computing device.
10. The computer-implemented method of claim 9 , wherein the plurality of data artifacts comprises one or more of (a) one or more schemas with their table names and associated column names, (b) index, trigger, and stored procedures associated with the schemas, (c) relationships between the different tables and databases, (d) table data, and (e) documentation, logs, performance and operational profile of datasets.
11. The computer-implemented method of claim 9 , wherein the plurality of code artifacts comprises source code and associated libraries.
12. The computer-implemented method of claim 9 , wherein the plurality of user interface artifacts comprises one or more of (a) user interface screens with natural language text, (b) user interface form objects and formatting, and (c) user interface modalities.
13. The computer-implemented method of claim 9 , wherein generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities comprises generating a first dependency graph of the set of dependency graphs based at least in part on one or more relationships between the extracted one or more entities of the data artifacts and the extracted one or more entities of the code artifacts.
14. The computer-implemented method of claim 13 , wherein generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities further comprises generating a second dependency graph of the set of dependency graphs based at least in part on one or more relationships between the extracted one or more entities of the data artifacts and the extracted one or more entities of the user interface artifacts.
15. The computer-implemented method of claim 14 , wherein generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities further comprises generating a third dependency graph of the set of dependency graphs based at least in part on one or more relationships between the extracted one or more entities of the code artifacts and the extracted one or more entities of the user interface artifacts.
16. The computer-implemented method of claim 9 , further comprising:
retrieving the data artifacts from at least one of a database and a file system; and
applying one of a data definition language operation and a data manipulation language operation to identify the one or more entities in each data artifact and determine one or more relationships between the one or more entities.
17. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:
extract one or more entities identified in a plurality of data artifacts based at least in part on one or more datasets;
extract one or more entities identified in a plurality of code artifacts based at least in part on the one or more datasets;
extract one or more entities identified in a plurality of user interface artifacts based at least in part on the one or more datasets;
generate a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities; and
perform one or more of a lexical analysis and a semantic analysis on the set of dependency graphs to identify a data domain of the one or more datasets.
18. The computer program product of claim 17 , wherein generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities comprises generating a first dependency graph of the set of dependency graphs based at least in part on one or more relationships between the extracted one or more entities of the data artifacts and the extracted one or more entities of the code artifacts.
19. The computer program product of claim 18 , wherein generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities further comprises generating a second dependency graph of the set of dependency graphs based at least in part on one or more relationships between the extracted one or more entities of the data artifacts and the extracted one or more entities of the user interface artifacts.
20. The computer program product of claim 19 , wherein generating a set of dependency graphs each based at least in part on one or more relationships among the respective extracted one or more entities further comprises generating a third dependency graph of the set of dependency graphs based at least in part on one or more relationships between the extracted one or more entities of the code artifacts and the extracted one or more entities of the user interface artifacts.