Transforming data metrics to maintain compatibility in an enterprise data warehouse
The present disclosure relates to systems, non-transitory computer-readable media, and methods for transforming data metrics to conform to a production schema for cross-domain uniformity. In particular, in one or more embodiments, the disclosed systems receive a metric having an initial schema, determine that the initial schema is inconsistent with a production schema, transform the metric to conform with the production schema, and store the metric in a standardized-schema database.
1. A method comprising:
receiving a first metric having a first initial schema and corresponding to a first data domain;
receiving a second metric having a second initial schema and corresponding to a second data domain;
determining that the first initial schema is inconsistent with a first production schema specific to the first data domain;
determining that the second initial schema is inconsistent with a second production schema specific to the second data domain, wherein the second production schema is different from the first production schema;
transforming the first metric to conform to the first production schema specific to the first data domain;
transforming the second metric to conform to the second production schema specific to the second data domain; and
storing, in a standardized-schema database, the first metric having the first production schema and the second metric having the second production schema.
2. The method of claim 1 , wherein receiving the first metric comprises retrieving the first metric from an ingestion database, a staging database, or a data-domain-specific database.
3. The method of claim 1 , further comprising utilizing executable instructions in a test schema to generate the first metric.
4. The method of claim 1 , further comprising verifying that the first metric follows the first production schema before storing the first metric in the standardized-schema database.
5. The method of claim 1 , further comprising receiving an additional metric having an additional initial schema differing from the first initial schema and corresponding to the first data domain.
6. The method of claim 5 , further comprising determining that the additional initial schema is inconsistent with the first production schema specific to the first data domain.
7. The method of claim 6 , further comprising transforming the additional metric to conform to the first production schema specific to the first data domain.
8. The method of claim 1 , wherein the standardized-schema database stores data having the first production schema specific to the first data domain and stores additional data having an additional production schema specific to an additional data domain.
9. The method of claim 1 , further comprising:
storing, in a data-domain-specific database, the first metric having the first production schema specific to the first data domain; and
storing, in an additional data-domain-specific database, an additional metric having an additional production schema specific to an additional data domain.
10. The method of claim 1 , further comprising generating an owner role, a writer role, and a reader role for the first initial schema.
11. A system comprising:
at least one processor; and
a non-transitory computer readable medium comprising instructions that, when executed by the at least one processor, cause the system to:
receive a first metric having a first initial schema and corresponding to a first data domain;
receive a second metric having a second initial schema and corresponding to a second data domain;
determine that the first initial schema is inconsistent with a first production schema specific to the first data domain;
determine that the second initial schema is inconsistent with a second production schema specific to the second data domain, wherein the second production schema is different from the first production schema;
transform the first metric to conform to the first production schema specific to the first data domain;
transform the second metric to conform to the second production schema specific to the second data domain; and
store, in a standardized-schema database, the first metric having the first production schema and the second metric having the second production schema.
12. The system of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the system to receive an additional metric having an additional initial schema differing from the first initial schema and corresponding to the first data domain.
13. The system of claim 12 , further comprising instructions that, when executed by the at least one processor, cause the system to determine that the additional initial schema is inconsistent with the first production schema specific to the first data domain.
14. The system of claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to transform the additional metric to conform to the first production schema specific to the first data domain.
15. The system of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the system to generate an owner role, a writer role, and a reader role for the first initial schema.
16. A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computing device to:
receive a first metric having a first initial schema and corresponding to a first data domain;
receive a second metric having a second initial schema and corresponding to a second data domain;
determine that the first initial schema is inconsistent with a first production schema specific to the first data domain;
determine that the second initial schema is inconsistent with a second production schema specific to the second data domain, wherein the second production schema is different from the first production schema;
transform the first metric to conform to the first production schema specific to the first data domain;
transform the second metric to conform to the second production schema specific to the second data domain; and
store, in a standardized-schema database, the first metric having the first production schema and the second metric having the second production schema.
17. The non-transitory computer readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computing device to receive the first metric by retrieving the first metric from an ingestion database, a staging database, or a data-domain-specific database.
18. The non-transitory computer readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computing device to utilize executable instructions in a test schema to generate the first metric.
19. The non-transitory computer readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computing device to verify that the first metric follows the first production schema before storing the first metric in the standardized-schema database.
20. The non-transitory computer readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computing device to store, in the standardized-schema database, an additional metric having an additional production schema specific to an additional data domain.