Schema generation for operational data governance

1 . A method implemented by a metadata management service, comprising:

receiving operational data generated by operations of a computer system;

analyzing the operational data to identify data structures within the operational data;

generating, based on the data structures, a schema definition that represents the operational data in a standardized format;

integrating the schema definition into the metadata management service;

receiving, via a continuous integration/continuous deployment (CI/CD) pipeline, source code for deployment to a production environment, wherein the source code contains operational data structures;

identifying, based on a mapping between source code components and approved schemas, data elements in the operational data structures that correspond to the schema definition;

determining, based on the schema definition, whether any classified data sensitivity levels associated with the identified data elements exceed a predetermined threshold; and

responsive to determining that the classified data sensitivity levels exceed the predetermined threshold, blocking the deployment of the source code to the production environment.

2 . The method of claim 1 , wherein analyzing the operational data to identify the data structures within the operational data comprises:

selecting, based on a schema type indicator, a parsing rule set, wherein the parsing rule set defines how to extract data elements from the operational data; and

applying the parsing rule set to the operational data to generate the schema definition.

3 . The method of claim 1 , further comprising:

mapping a field in the schema definition to a standardized term in a centralized dictionary.

4 . The method of claim 3 , wherein the standardized term is associated with a sensitivity classification.

5 . The method of claim 1 , wherein integrating the schema definition into the metadata management service comprises:

submitting the schema definition to a governance workflow for classification and approval; and

upon receiving the approval via the governance workflow, integrating the schema definition into the metadata management service.

6 . The method of claim 5 , further comprising:

providing a user interface for editing the schema definition before submission to the governance workflow.

7 . The method of claim 1 , wherein the operational data comprises one of system logs generated for debugging or monitoring, message queue payloads, client-side memory logs containing debugging information, or server-side multimedia router logs.

8 . The method of claim 1 , wherein analyzing the operational data to identify the data structures within the operational data comprises:

extracting implicit data structures from unstructured or semi-structured operational data, wherein the implicit data structures comprise log message definitions or telemetry collection mechanisms within the source code; and

converting the implicit data structures into definitions of tables and fields that represent the data structures.

9 . A system, comprising:

a memory subsystem; and

processing circuitry, the processing circuitry configured to execute instructions of a metadata management service and stored in the memory subsystem to:

receive operational data generated by operations of a computer system;

analyze the operational data to identify data structures within the operational data;

generate, based on the data structures, a schema definition that represents the operational data in a standardized format;

integrate the schema definition into the metadata management service;

receive, via a continuous integration/continuous deployment (CI/CD) pipeline, source code for deployment to a production environment, wherein the source code contains operational data structures;

identify, based on a mapping between source code components and approved schemas, data elements in the operational data structures that correspond to the schema definition;

determine, based on the schema definition, whether any classified data sensitivity levels associated with the identified data elements exceed a predetermined threshold; and

responsive to determining that the classified data sensitivity levels exceed the predetermined threshold, block the deployment of the source code to the production environment.

10 . The system of claim 9 , wherein the instructions to receive the operational data comprise instructions to:

receive source code that generates the operational data during their operation.

11 . The system of claim 9 , wherein the instructions to analyze the operational data comprise instructions to:

analyze log message implementations within source code to identify data elements generated during execution.

12 . The system of claim 9 , wherein the instructions to analyze the operational data comprise instructions to:

parse comma-separated values within log messages to identify data fields and their relationships.

13 . The system of claim 9 , wherein the instructions to generate the schema definition comprise instructions to:

generate definitions of tables and fields that represent the data structures.

14 . The system of claim 9 , wherein the instructions to generate the schema definition comprise instructions to:

convert the operational data into a standardized JSON format.

15 . The system of claim 9 , wherein the processing circuitry is further configured to execute instructions stored in the memory subsystem to:

receive configuration parameters that define rules for analyzing new types of operational data formats.

16 . The system of claim 9 , wherein the processing circuitry is further configured to execute instructions stored in the memory subsystem to:

submit the schema definition to a privacy reviewer for classification of data sensitivity levels.

17 . One or more non-transitory computer readable media storing instructions operable to cause one or more processors to perform operations of a metadata management service, the operations comprising:

receiving operational data generated by operations of a computer system;

analyzing the operational data to identify data structures within the operational data;

generating, based on the data structures, a schema definition that represents the operational data in a standardized format;

integrating the schema definition into the metadata management service;

receiving, via a continuous integration/continuous deployment (CI/CD) pipeline, source code for deployment to a production environment, wherein the source code contains operational data structures;

identifying, based on a mapping between source code components and approved schemas, data elements in the operational data structures that correspond to the schema definition;

determining, based on the schema definition, whether any classified data sensitivity levels associated with the identified data elements exceed a predetermined threshold; and

responsive to determining that the classified data sensitivity levels exceed the predetermined threshold, blocking the deployment of the source code to the production environment.

18 . The one or more non-transitory computer readable media of claim 17 , wherein integrating the schema definition comprises:

applying access control policies based on classified data sensitivity levels.

19 . The one or more non-transitory computer readable media of claim 17 , wherein integrating the schema definition comprises:

linking the schema definition to a data dictionary containing approved data terms.

20 . The one or more non-transitory computer readable media of claim 17 , the operations further comprising:

automatically suggesting matches between data elements in the schema definition and pre-approved terms in a data dictionary.