Systems and methods for automated data dictionary generation and validation

1. A method for automated data dictionary validation, the method comprising:

automatically generating, by data dictionary generation circuitry, a data dictionary based on one or more reference term tokens of a database query, wherein automatically generating the data dictionary comprises:

parsing, by the data dictionary generation circuitry, the database query, and

identifying, by the data dictionary generation circuitry and based on the parsing of the database query, the one or more reference term tokens;

validating, by data dictionary validation circuitry, the data dictionary, wherein validating the data dictionary includes:

generating, by a skeleton query generator and based on the database query and the data dictionary, a skeleton query, wherein generating the skeleton query comprises:

identifying, by the skeleton query generator, a set of native term tokens in the database query, wherein the one or more native term tokens are native to a programming language used to generate the database query,

removing, by the skeleton query generator, the set of native term tokens from the database query, and

removing a respective reference term token from the database query based on an identification of the respective reference term token in the data dictionary;

determining, by the data dictionary validation circuitry, whether the skeleton query is empty, and

determining, by the data dictionary validation circuitry and based on the determination as to whether the skeleton query is empty, a validation status of the data dictionary, wherein the validation status indicates a successful validation of the data dictionary in an instance in which the skeleton query is determined to be empty and an unsuccessful validation of the data dictionary in an instance in which the skeleton query comprises at least one of the one or more reference term tokens; and

causing, by the data dictionary validation circuitry, presentation of the validation status.

2. An apparatus for automated data dictionary validation comprising:

a processor; and

data dictionary generation circuitry configured, with the processor, to:

automatically generate a data dictionary based on one or more reference term tokens of a database query, wherein automatically generating the data dictionary comprises:

parsing the database query, and

identifying, based on the parsing of the database query, the one or more reference term tokens;

a skeleton query generator configured, with the processor, to:

generate, based on the database query and the data dictionary, a skeleton query, wherein generating the skeleton query comprises:

identifying a set of native term tokens in the database query, wherein the one or more native term tokens are native to a programming language used to generate the database query,

removing the set of native term tokens from the database query, and

removing a respective reference term token from the database query based on an identification of the respective reference term token in the data dictionary; and

data dictionary validation circuitry configured, with the processor, to:

validate the data dictionary, wherein validating the data dictionary includes:

determining whether the skeleton query is empty, and

determining, based on the determination as to whether the skeleton query is empty, a validation status of the data dictionary, wherein the validation status indicates a successful validation of the data dictionary in an instance in which the skeleton query is determined to be empty and an unsuccessful validation of the data dictionary in an instance in which the skeleton query comprises at least one of the one or more reference term tokens; and

cause presentation of the validation status.

3. A computer program product for automated data dictionary validation, the computer program product comprising at least one non-transitory computer-readable storage medium storing software instructions that, when executed, cause an apparatus to:

automatically generate a data dictionary based on one or more reference term tokens of a database query, wherein automatically generating the data dictionary comprises:—

parsing, by the data dictionary generation circuitry, the database query, and

identifying, by the data dictionary generation circuitry and based on the parsing of the database query, the one or more reference term tokens; and

validate the data dictionary, wherein validating the data dictionary includes:

generating, based on the database query and the data dictionary, a skeleton query, wherein generating the skeleton query comprises:

identifying a set of native term tokens in the database query, wherein the one or more native term tokens are native to a programming language used to generate the database query,

removing the set of native term tokens from the database query, and

removing a respective reference term token from the database query based on an identification of the respective reference term token in the data dictionary;

determining whether the skeleton query is empty, and

determining, based on the skeleton query, a validation status of the data dictionary, wherein the validation status indicates a successful validation of the data dictionary in an instance in which the skeleton query is determined to be empty and an unsuccessful validation of the data dictionary in an instance in which the skeleton query comprises at least one of the one or more reference term tokens; and

cause presentation of the validation status.

4. The method of claim 1 , further comprising:

causing, by the data dictionary validation circuitry, presentation of the skeleton query.

5. The apparatus of claim 2 , wherein the data dictionary validation circuitry is further configured, with the processor, to:

cause presentation of the skeleton query.

6. The computer program product of claim 3 , further comprising software instructions that, when executed, cause the apparatus to:

cause presentation of the skeleton query.

7. The method of claim 1 , wherein the database query is a Structured Query Language (SQL) query.

8. The method of claim 1 , further comprising, in response to the validation status indicating a successful validation of the data dictionary:

validating, by the data dictionary validation circuitry, a second data dictionary, wherein validating the second data dictionary comprises:

comparing, by the data dictionary validation circuitry, the data dictionary to the second data dictionary to identify a duplicate set of reference term tokens including reference term tokens included in both the data dictionary and the second data dictionary, and

performing, by the data dictionary validation circuitry, a removal process comprising removal of the duplicate set of reference term tokens from the second data dictionary.

9. The method of claim 8 , wherein the second data dictionary is a manually-created data dictionary that was generated prior to generation of the data dictionary.

10. The method of claim 1 , wherein automatically generating the data dictionary further comprises:

converting, by the data dictionary generation circuitry, an alias indicated by a reference term token of the one or more reference term tokens in the database query to a data source name; and

storing, by the data dictionary generation circuitry, the data source name in a data-source referencing field of the data dictionary.

11. The method of claim 1 , wherein automatically generating the data dictionary further comprises:

identifying, by the data dictionary generation circuitry, that a reference term token of the one or more reference term tokens in the database query does not include a data source name; and

storing, by the data dictionary generation circuitry, an indication of a missing data source name in a data-source referencing field of the data dictionary.

12. The apparatus of claim 2 , wherein the database query is a Structured Query Language (SQL) query.

13. The apparatus of claim 2 , wherein the data dictionary validation circuitry is further configured, with the processor, to, in response to the validation status indicating a successful validation of the data dictionary:

validate a second data dictionary, wherein validating the second data dictionary comprises:

comparing the data dictionary to the second data dictionary to identify a duplicate set of reference term tokens including reference term tokens included in both the data dictionary and the second data dictionary, and

performing a removal process comprising removal of the duplicate set of reference term tokens from the second data dictionary.

14. The apparatus of claim 13 , wherein the second data dictionary is a manually-created data dictionary that was generated prior to generation of the data dictionary.

15. The apparatus of claim 1 , wherein automatically generating the data dictionary further comprises:

converting an alias indicated by a reference term token of the one or more reference term tokens in the database query to a data source name; and

storing the data source name in a data-source referencing field of the data dictionary.

16. The apparatus of claim 1 , wherein automatically generating the data dictionary further comprises:

identifying that a reference term token of the one or more reference term tokens in the database query does not include a data source name; and

storing an indication of a missing data source name in a data-source referencing field of the data dictionary.

17. The computer program product of claim 3 , wherein the data dictionary validation circuitry is further configured to, in response to the validation status indicating a successful validation of the data dictionary:

validate, by the data dictionary validation circuitry, a second data dictionary, wherein validating the second data dictionary comprises:

comparing the data dictionary to the second data dictionary to identify a duplicate set of reference term tokens including reference term tokens included in both the data dictionary and the second data dictionary, and

performing a removal process comprising removal of the duplicate set of reference term tokens from the second data dictionary.

18. The computer program product of claim 17 , wherein the second data dictionary is a manually-created data dictionary that was generated prior to generation of the data dictionary.

19. The computer program product of claim 1 , wherein automatically generating the data dictionary further comprises:

converting an alias indicated by a reference term token of the one or more reference term tokens in the database query to a data source name; and

storing the data source name in a data-source referencing field of the data dictionary.

20. The computer program product of claim 1 , wherein automatically generating the data dictionary further comprises:

identifying that a reference term token of the one or more reference term tokens in the database query does not include a data source name; and

storing an indication of a missing data source name in a data-source referencing field of the data dictionary.