Interactive data processing system failure management using hidden knowledge from predictive models
Methods and systems for managing data processing systems are disclosed. A data processing system may include and depend on the operation of hardware and/or software components. Inference models may be implemented to predict future system infrastructure outcomes (e.g., component failures) using information recorded in logs that reflect the operation of the components. However, the models may be complex “black boxes” and may generate critical outcome predictions for downstream consumers without explanations of how the predictions are determined, resulting in downstream consumers having low confidence in the predictions. Therefore, hidden knowledge (e.g., structured knowledge attributes) of the models may be extracted and/or used to understand the underlying processes that the models use to predict the system infrastructure outcomes. The hidden knowledge may be provided for interactively managing data processing system(s) failures in order to increase the likelihood of preventing and/or mitigating future data processing system failures.
1 . A method for managing failures of data processing systems, comprising and by a data processing system manager configured to manage the data processing systems:
obtaining a data request, from a requestor, for data stored in a structured knowledge repository, the data comprising structured knowledge attributes extracted from an architecture of a trained machine learning model, wherein the architecture of the trained machine learning model is hidden;
determining, based on the data request, a type of components of the data processing system;
filtering the structured knowledge attributes based on the type of components to obtain filtered structured knowledge attributes;
generating one or more customized user response prompts using the data and generic response prompts stored in a sample prompt repository;
obtaining a response to the data request using the one or more customized user response prompts;
providing the response to the requestor, through an interactive user interface through which the data request was received, to service the data request, wherein the response comprises a failure prediction and a portion of the filtered structured knowledge attributes that provide visibility for the requestor into understanding how and why the trained machine learning model generated the failure prediction in a manner that the trained machine learning model generated the failure prediction without the requestor having direct accessibility to the architecture, the failure prediction being one of inferences generated by the machine learning model, wherein the failure prediction is associated with the type of components; and
troubleshooting a component having the type of components according to the failure prediction.
2 . The method of claim 1 , wherein the structured knowledge attributes are usable to manage an indication of failure for a data processing system of the data processing systems.
3 . The method of claim 2 , wherein the one or more customized user response prompts are generated using few shot learning techniques.
4 . The method of claim 3 , further comprising by the data processing system manager:
refining the data request to obtain a refined data request, wherein the refining comprises:
obtaining a user intention from the data request; and
refining the data request based on the user intention and the structured knowledge attributes stored in the structured knowledge repository,
wherein the one or more customized user response prompts is further generated using the refined data request.
5 . The method of claim 4 , further comprising by the data processing system manager:
obtaining user preference data from a local domain context repository, wherein the user preference data is associated with the requestor,
wherein the one or more customized user response prompts is further generated using the user preference data.
6 . The method of claim 2 , further comprising by the data processing system manager:
prior to generating the response:
identifying an occurrence of the indication of failure for the data processing system; and
based on the occurrence, using an inference model to obtain an indication of a root cause for the failure, the structured knowledge repository being based, at least in part, on the inference model and logs on which the inference model is based, the inference model being the trained machine learning model that generates the failure prediction, and the indication of the root cause being specified in the failure prediction.
7 . The method of claim 6 , further comprising by the data processing system manager:
after providing the response:
assessing a likelihood of the root cause being accurate; and
in an instance of the assessing where the likelihood meets a threshold:
identifying at least one remediation action based on the root cause; and
performing the at least one remediation action to obtain an updated data processing system to attempt to remediate the failure.
8 . A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor of a data processing system manager configured to manage data processing systems, cause the data processing system manager to perform operations for managing failures of the data processing systems, the operations comprising:
obtaining a data request, from a requestor, for data stored in a structured knowledge repository;
generating one or more customized user response prompts using the data and generic response prompts stored in a sample prompt repository, the data comprising structured knowledge attributes extracted from an architecture of a trained machine learning model, wherein the architecture of the trained machine learning model is hidden,
determining, based on the data request, a type of components of the data processing systems;
filtering the structured knowledge attributes based on the type of components to obtain filtered structured knowledge attributes;
obtaining a response to the data request using the one or more customized user response prompts;
providing the response to the requestor, through an interactive user interface through which the data request was received, to service the data request, wherein the response comprises a failure prediction and a portion of the filtered structured knowledge attributes that provide visibility for the requestor into understanding how and why the trained machine learning model generated the failure prediction in a manner that the trained machine learning model generated the failure prediction without the requestor having direct accessibility to the architecture, the failure prediction being one of inferences generated by the machine learning model, wherein the failure prediction is associated with the type of components; and
troubleshoot a component having the type of components according to the failure prediction.
9 . The non-transitory machine-readable medium of claim 8 , wherein the structured knowledge attributes are usable to manage an indication of failure for a data processing system of the data processing systems.
10 . The non-transitory machine-readable medium of claim 9 , wherein the one or more customized user response prompts are generated using few shot learning techniques.
11 . The non-transitory machine-readable medium of claim 10 , wherein the operations further comprise:
refining the data request to obtain a refined data request, wherein the refining comprises:
obtaining a user intention from the data request; and
refining the data request based on the user intention and the structured knowledge attributes stored in the structured knowledge repository,
wherein the one or more customized user response prompts is further generated using the refined data request.
12 . The non-transitory machine-readable medium of claim 11 , wherein the operations further comprise:
obtaining user preference data from a local domain context repository, wherein the user preference data is associated with the requestor,
wherein the one or more customized user response prompts is further generated using the user preference data.
13 . A data processing system manager, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the data processing system manager to perform operations for managing failures of data processing systems, the operations comprising:
obtaining a data request, from a requestor, for data stored in a structured knowledge repository, the data comprising structured knowledge attributes extracted from an architecture of a trained machine learning model, wherein the architecture of the trained machine learning model is hidden;
determining, based on the data request, a type of components of the data processing system;
filtering the structured knowledge attributes based on the type of components to obtain filtered structured knowledge attributes;
generating one or more customized user response prompts using the data and generic response prompts stored in a sample prompt repository;
obtaining a response to the data request using the one or more customized user response prompts;
providing the response to the requestor, through an interactive user interface through which the data request was received, to service the data request, wherein the response comprises a failure prediction and a portion of the filtered structured knowledge attributes that provide visibility for the requestor into understanding how and why the trained machine learning model generated the failure prediction in a manner that the trained machine learning model generated the failure prediction without the requestor having direct accessibility to the architecture, the failure prediction being one of inferences generated by the machine learning model, wherein the failure prediction is associated with the type of components; and
troubleshoot a component having the type of components according to the failure prediction.
14 . The data processing system manager of claim 13 , wherein the structured knowledge attributes are usable to manage an indication failure for a data processing system of the data processing systems.
15 . The data processing system manager of claim 14 , wherein the one or more customized user response prompts are generated using few shot learning techniques.
16 . The data processing system manager of claim 15 , wherein the operations further comprise:
refining the data request to obtain a refined data request, wherein the refining comprises:
obtaining a user intention from the data request; and
refining the data request based on the user intention and the structured knowledge attributes stored in the structured knowledge repository,
wherein the one or more customized user response prompts is further generated using the refined data request.
17 . The method of claim 1 , further comprising and by the data processing system manager prior to obtaining the data request:
extracting the structured knowledge attributes from the architecture of the trained machine learning model, the trained machine learning model being hosted by the data processing system manager; and
storing the structured knowledge attributes extracted from the architecture of the trained machine learning model into the structured knowledge repository.
18 . The method of claim 1 , wherein the architecture of the trained machine learning model comprises at least information regarding one or more other trained machine learning models that were used to train the trained machine learning model and regarding respective other architectures of each of the one or more other trained machine learning models.
19 . The method of claim 1 , wherein the one or more structured knowledge attributes comprise parameters on which the trained machine learning model is trained to generate the inferences.
20 . The method of claim 19 , wherein the one or more structured knowledge attributes further comprise relationships between components of the trained machine learning model, the components comprising at least input features of data ingested into the trained machine learning model, the inferences generated by the trained machine learning model, and one or more rules followed by the trained machine learning model to generate the inferences.