Autonomous AI agent training using time-based simulation models
The disclosed method and system involves training an autonomous AI agent using simulation models that incorporate temporal progression in decision-making. The method begins with ingesting system operation data, which includes receiving historical data in a time series sequence and identifying observation sensor and action variables. The data is checked for simulation readiness, and any time gaps in the sequence are addressed by segmenting the data into batches to exclude these gaps. The segmented data is then mapped to a framework for time-based models that account for temporal progression. These models are trained using the mapped data, and a simulation of the system is generated based on the trained models. The trained time-based models are discretized and shaped to fit a decision-making process, which is then used to train the autonomous AI agent. This approach ensures the agent's decision-making is informed by temporal dynamics within the system.
1 . A method for training an autonomous artificial intelligence (AI) agent using simulation models that consider a progression of time in decision-making processes, the method comprising:
ingesting data for operation of a system, the ingesting comprising:
receiving, via one or more processors, data characterizing a system that includes historical data in a time series sequence collected by sensors monitoring the system;
identifying, by the one or more processors, observation sensor variables and action variables from the received data;
checking, by the one or more processors, whether the data meets minimum requirements for simulation, including verifying time frequency consistency and sensor data quality;
determining, by the one or more processors, whether there are one or more time gaps in the time series sequence that would disrupt temporal continuity of the data;
when there are one or more time gaps in the time series sequence, segmenting, by the one or more processors, the received data into temporally continuous batches to preserve temporal relationships while omitting the time gaps in the time series sequence; and
mapping, by the one or more processors, the segmented data to a framework for one or more machine learning time-based models that consider a progression of time in the decision-making processes;
training, by the one or more processors, the one or more machine learning time-based models using the mapped data to generate trained models that predict future system states based on current and historical states and actions;
generating, by the one or more processors, a simulation for the system based on the one or more trained time-based models, wherein the one or more trained time-based models are discretized and shaped to a decision-making process to create a virtual environment that mimics temporal behavior of the system;
training the autonomous AI agent using the simulation, wherein the trained autonomous AI agent learns to make decisions that account for temporal dynamics of the system; and
deploying the trained autonomous AI agent to interface with the system in real-time, wherein the trained autonomous AI agent controls hardware or software of the system based on decisions informed by the temporal dynamics learned from the simulation.
2 . The method of claim 1 , wherein checking whether the data meets minimum requirements for simulation comprises:
checking for missing data or non-numeric data;
checking whether time frequency of time series sequence is consistent and equally spaced;
checking whether identified sensor variables and action variables are present in the data;
checking whether sensor data is outside of defined constraints; and
checking for sensor quality issues.
3 . The method of claim 1 , wherein mapping the data to a framework for one or more models that consider a progression of time in decision-making processes comprises:
establishing a present state of the system (S t ) and a current action (A t ) for machine learning models based on the data;
establishing one or more historical states (S t minus time increment) and one or more historical actions (A t minus time increment) for machine learning models based on the data; and
establishing a future state (S t +1) and Action (A t +1) to be determined by a machine learning model.
4 . The method of claim 1 , wherein the one or more machine learning time-based models comprise a Markov decision process (MDP) in a discretized simulation.
5 . The method of claim 1 , wherein training one or more machine learning time-based models comprises:
training a machine learning model for each sensor variable in the received data; and
evaluating the trained machine learning model by:
testing sequence predictions against original data;
checking correlations;
checking feature importance;
checking interpolation and extrapolation; and
performing error propagation analysis to determine a prediction horizon.
6 . The method of claim 5 , wherein evaluating the trained machine learning model further comprises:
performing error propagation analysis over different time horizons to identify an optimal time horizon with a lowest error propagation when comparing simulation predictions to the received data.
7 . The method of claim 1 , wherein generating a simulation for the system from the one or more trained models comprises:
providing a trained model for each sensor value in the system; and
providing constraints and initial values for each sensor variable.
8 . The method of claim 1 , wherein training the autonomous AI agent using the simulation comprises:
providing the autonomous AI agent;
operating the autonomous AI agent in the simulation; and
providing feedback on the operation of the autonomous AI agent in the simulation.
9 . The method of claim 1 , wherein the one or more time-based models comprise at least one of a recurrent neural network (RNN), an XGBoost Regressor, a long short-term memory (LSTM) network, a generalized linear model (GLM), or a gated recurrent unit (GRU) network.
10 . The method of claim 1 , wherein mapping the segmented data to the framework comprises aligning the segmented training data with a predefined input format for the one or more time-based models.
11 . The method of claim 1 , wherein training the autonomous AI agent comprises utilizing one or more of reinforcement learning, proportional-integral-derivative (PID) controllers, model predictive control (MPC) controllers, or a combination of thereof, using the simulation as an environment.
12 . The method of claim 1 , wherein segmenting the received data into temporally continuous batches improves accuracy of the trained time-based models by preventing the models from learning incorrect temporal relationships caused by discontinuities in the time series sequence.
13 . The method of claim 1 , wherein mapping the segmented training data to the framework comprises structuring the segmented data such that each data point includes present states, present actions, historical states, and historical actions across multiple time increments, thereby preserving temporal dependencies between states and actions for the trained models.
14 . The method of claim 1 , wherein the trained autonomous AI agent interfaces with a real-world system to control hardware or software based on decisions informed by the temporal dynamics learned from the simulation.
15 . The method of claim 1 , further comprising:
deploying the trained autonomous AI agent to control the system in real-time, wherein the autonomous AI agent applies learned temporal relationships to predict system behavior and optimize control actions.
16 . The method of claim 1 , wherein the system comprises at least one of a manufacturing system, an engineering system, or a logistics system, and wherein the trained autonomous AI agent controls operation of the system based on predictions of future system states derived from the trained time-based models.
17 . The method of claim 1 , wherein generating the simulation comprises creating a virtual environment that receives action inputs and produces state outputs that reflect how the system evolves over time based on the trained time-based models.
18 . The method of claim 1 , wherein the one or more time-based models comprise at least one of a recurrent neural network (RNN), an XGBoost Regressor, a long short-term memory (LSTM) network, a generalized linear model (GLM), or a gated recurrent unit (GRU) network.
19 . The method of claim 1 , wherein mapping the segmented data to the framework comprises aligning the segmented training data with a predefined input format for the one or more time-based models.
20 . The method of claim 1 , wherein training the autonomous AI agent comprises utilizing one or more of reinforcement learning, proportional-integral-derivative (PID) controllers, model predictive control (MPC) controllers, or a combination thereof, using the simulation as an environment.
21 . A training system for training an autonomous artificial intelligence (AI) agent using models that consider a progression of time in decision-making processes, the training system comprising:
one or more processors;
a data quality check module configured executed by the one or more processors to:
receive data characterizing a system that includes historical data in a time series sequence collected from sensors monitoring the system;
identify observation sensor variables and action variables from the received data;
check whether the data meets minimum requirements for simulation;
determine whether there are one or more time gaps in the time series sequence;
when there are one or more time gaps in the time series sequence, segment the received data into temporally continuous batches to preserve temporal relationships while omitting the one or more time gaps in the time series sequence; and
map the segmented data to a framework for one or more machine learning time-based models that consider a progression of time in decision-making processes;
a machine learning model training module configured executed by the one or more processors to:
train a machine learning model for each sensor variable in the data to predict future values based on current and historical states and actions; and
evaluate the trained machine learning model;
a simulation generation module configured executed by the one or more processors to:
provide a trained model for each sensor value in the system; and
provide constraints and initial values for each trained model to generate a simulation that mimics temporal behavior of the system; and
an agent training module configured executed by the one or more processors to train the autonomous AI agent using the simulation to learn decisions that account for temporal dynamics of the system and
deploy the trained autonomous AI agent to interface with the system in real-time, wherein the trained autonomous AI agent controls hardware or software of the system based on decisions informed by the temporal dynamics learned from the simulation.
22 . The system of claim 21 , wherein checking whether the data meets minimum requirements for simulation comprises:
checking for missing data or non-numeric data;
checking whether time frequency of time series sequence is consistent and equally spaced;
checking whether identified observation sensor variables and action variables are present in the data;
checking whether sensor data is outside of defined constraints; and
checking for sensor quality issues.
23 . The system of claim 21 , wherein the agent training module is further configured executed by the one or more processors to:
operate the autonomous AI agent in the simulation; and
provide feedback on operation of the autonomous AI agent in the simulation.
24 . The system of claim 21 , wherein the simulation generation module is further configured executed by the one or more processors to provide physical constraints for each sensor and action variable in the data, defining boundaries within which the simulation operates.
25 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving data characterizing a system that includes historical data in a time series sequence collected from sensors monitoring the system;
identifying observation sensor variables and action variables from the received data; checking whether the data meets minimum requirements for simulation;
determining whether there are one or more time gaps in the time series sequence; when there are one or more time gaps in the time series sequence, segmenting the received data into temporally continuous batches to preserve temporal relationships while omitting the time gaps in the time series sequence;
mapping the segmented data to a framework for one or more machine learning time-based models that consider a progression of time in decision-making processes;
training the one or more machine learning time-based models using the mapped data to generate trained models that predict future system states based on current and historical states and actions;
generating a simulation for the system based on the one or more trained time-based models to create a virtual environment that mimics temporal behavior of the system;
training an autonomous artificial intelligence (AI) agent using the simulation, wherein the trained autonomous AI agent learns to make decisions that account for temporal dynamics of the system; and
deploying the trained autonomous AI agent to interface with the system in real-time, wherein the trained autonomous AI agent controls hardware or software of the system based on decisions informed by the temporal dynamics learned from the simulation.
26 . The non-transitory computer-readable medium of claim 25 , wherein the operations further comprise:
training a machine learning model for each sensor variable in the received data; and
evaluating each trained machine learning model by testing sequence predictions against original data and checking correlations between sensor variables and actions.