Granted Patent B2

US 12,475,019 · App. 16/973,093 · Granted Nov 18, 2025

Inventors: Hiroki Ikeuchi (Tokyo, JP); Akio Watanabe (Tokyo, JP); Takehiro Kawata (Tokyo, JP)

Assignee: NTT, Inc.

G06F11/3476G06F11/07G06F11/3438G06N7/00G06N20/00

Abstract

Claims (43)

1 . A system state estimation apparatus to estimate a system state in a real environment system that outputs log data upon executing a user action, the system state estimation apparatus comprising:

a memory; and

a processor configured to:

acquire, from a database, training data that comprises prior data output by the real environment system from prior actions performed by one or more user instructions under different system failure conditions;

generate a set of prior feature vectors that represent the acquired training data, wherein each prior feature vector of the set of prior feature vectors comprises one or more prior user actions and one or more prior corresponding system states;

train, using the generated set of prior feature vectors, a stochastic model that generates a likelihood of systems states, wherein the stochastic model represents a likelihood of which feature vectors occur in response to one or more respective user actions being taken upon certain system failures;

generate, in a cloud computing environment, a simulation that models narrowing candidate causes of a system failure to reduce processing load on the system and to isolate a final cause of system failure by predicting a number of user actions required to be executed to determine the final cause of the system failure using the trained stochastic model, the simulation being configured to:

determine, from among a plurality of user actions, a user action that is most likely to reduce the plurality of user actions that caused an unknown system state using (i) the trained stochastic model and (ii) a probability distribution that defines probabilities of different system states being the unknown system state;

execute the determined user action on the real environment system;

in response to executing the determined user action, obtain, from the real environment system, feedback data that includes results of the determined user action on the real environment system, wherein the feedback data includes non-binary data;

update the probability distribution based on the obtained feedback data including the non-binary data; and

determine, based on the updated probability distribution, whether any of the different system states has a probability that meets a threshold value;

repeat the narrowing of the candidate causes of the system failure in the simulation until a determination is made that at least one of the different system states has the probability that meets the threshold value, wherein repeating of the narrowing simulation is performed using the updated probability distribution and the trained stochastic model and wherein repeating of the narrowing simulation comprises excluding previously executed user actions from the plurality of user actions;

in response to determining that a particular system state among the different system states has the probability that meets the threshold value, terminate the repeated simulation; and

output the particular system state from the simulation as the unknown state to a user interface.

2 . The system state estimation apparatus of claim 1 , wherein the processor is configured to determine the respective user action that minimizes entropy of the probability distribution of the unknown system state.

3 . The system state estimation apparatus of claim 1 , wherein the processor is configured to determine the respective user action that maximizes a value obtained by inputting a candidate system states and the respective user action into a function determined by Q-learning, as the respective user action that contributes to the unknown system state.

4 . A system state estimation method executed by a system state estimation apparatus to estimate a system state in a system that outputs log data upon executing a user action, the system state estimation method comprising:

acquiring, from a database, training data that comprises prior data output by a real environment system from prior actions performed by one or more user instructions under different system failure conditions;

generating a set of prior feature vectors that represent the acquired training data, wherein each prior feature vector of the set of prior feature vectors comprises one or more prior user actions and one or more prior corresponding system states;

training, using the generated set of prior feature vectors, a stochastic model that generates a likelihood of systems states, wherein the stochastic model represents a likelihood of which feature vectors occur in response to one or more respective user actions being taken upon certain system failures;

generating, in a cloud computing environment, a simulation that models narrowing candidate causes of a system failure to reduce processing load on the system and to isolate a final cause of system failure by predicting a number of user actions required to be executed to determine the final cause of the system failure using the trained stochastic model, the simulation being configured to:

determining, from among a plurality of user actions, a user action that is most likely to reduce the plurality of user actions that caused an unknown system state using (i) the trained stochastic model and (ii) a probability distribution that defines probabilities of different system states being the unknown system state;

executing the determined user action on the real environment system;

in response to executing the determined user action, obtaining, from the real environment system, feedback data that includes results of the determined user action on the real environment system, wherein the feedback data includes non-binary data;

updating the probability distribution based on the obtained feedback data including the non-binary data; and

determining, based on the updated probability distribution, whether any of the different system states has a probability that meets a threshold value;

repeating the narrowing of the candidate causes of the system failure in the simulation until a determination is made that at least one of the different system states has the probability that meets the threshold value, wherein repeating of the narrowing simulation is performed using the updated probability distribution and the trained stochastic model and wherein repeating of the narrowing simulation comprises excluding previously executed user actions from the plurality of user actions;

in response to determining that a particular system state among the different system states has the probability that meets the threshold value, terminate the narrowing of the repeated simulations; and

outputting the particular system state from the simulation as the unknown state to a user interface.

5 . A non-transitory computer-readable recording medium having computer-readable instructions stored thereon, which when executed, causes a computer to execute a process of system state estimation, the process comprising:

acquiring, from a database, training data that comprises prior data output by a real environment system from prior actions performed by one or more user instructions under different system failure conditions;

generating a set of prior feature vectors that represent the acquired training data, wherein each prior feature vector of the set of prior feature vectors comprises one or more prior user actions and one or more prior corresponding system states;

training, using the generated set of prior feature vectors, a stochastic model that generates a likelihood of systems states, wherein the stochastic model represents a likelihood of which feature vectors occur in response to one or more respective user actions being taken upon certain system failures;

generating, in a cloud computing environment, a simulation that models narrowing candidate causes of a system failure to reduce processing load on the system and to isolate a final cause of system failure by predicting a number of user actions required to be executed to determine the final cause of the system failure using the trained stochastic model, the simulation being configured to:

determining, from among a plurality of user actions, a user action that is most likely to reduce the plurality of user actions that caused an unknown system state using (i) the trained stochastic model and (ii) a probability distribution that defines probabilities of different system states being the unknown system state;

executing the determined user action on the real environment system;

in response to executing the determined user action, obtaining, from the real environment system, feedback data that includes results of the determined user action on the real environment system, wherein the feedback data includes non-binary data;

updating the probability distribution based on the obtained feedback data including the non-binary data; and

determining, based on the updated probability distribution, whether any of the different system states has a probability that meets a threshold value;

repeating the narrowing of the candidate causes of the system failure in the simulation until a determination is made that at least one of the different system states has the probability that meets the threshold value, wherein repeating of the narrowing simulation is performed using the updated probability distribution and the trained stochastic model and wherein repeating of the narrowing simulation comprises excluding previously executed user actions from the plurality of user actions;

in response to determining that a particular system state among the different system states has the probability that meets the threshold value, terminate the narrowing of the repeated simulations; and

outputting the particular system state from the simulation as the unknown state to a user interface.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (23)