Detecting malfunctioning artificial intelligence agents
The present disclosure provides a malfunction interdiction system with monitoring, detection, and response capabilities specifically designed for distributed AI environments. The system may first detect any AI agents that are malfunctioning and then remediate any malfunctions via quarantine and/or AI agent termination. Upon detection of a malfunction, the system may automatically isolate the compromised agent through a quarantine process. The system may alternatively or additionally, initiate graduated deactivation procedures ranging from partial restrictions to full deactivation.
1 . One or more non-transitory computer-readable storage media having instructions stored thereon, the instructions causing one or more processors to perform operations comprising:
receiving, from a first artificial intelligence agent of a plurality of artificial intelligence agents, a response to a request,
wherein each artificial intelligence agent of the plurality of artificial intelligence agents is associated with a corresponding computer-executable operation set configured to be autonomously executed by a corresponding artificial intelligence agent on a software application set;
retrieving (1) a plurality of agent parameters associated with the first artificial intelligence agent, (2) an agent profile associated with the first artificial intelligence agent, and (3) a plurality of sets of historical agent parameters,
wherein the plurality of agent parameters comprises one or more of communication parameters, resource parameters or timing parameters, and
wherein the agent profile comprises one or more of a communications profile, a resource profile, or a timing profile;
inputting the plurality of agent parameters and the agent profile into a machine learning model to obtain a behavior parameter that indicates whether the plurality of agent parameters match the agent profile,
wherein the machine learning model is trained, using historical communication data, historical resource data, and historical timing data associated with the first artificial intelligence agent, to recognize patterns within the plurality of agent parameters to match with corresponding patterns within the agent profile;
inputting the plurality of agent parameters into an ensemble anomaly detection model to obtain an anomaly parameter whether the plurality of agent parameters indicate an anomaly with the response,
wherein the ensemble anomaly detection model is trained to identify anomalies between the plurality of agent parameters and timeseries data associated with the plurality of sets of historical agent parameters;
determining based on the behavior parameter and the anomaly parameter whether the first artificial intelligence agent is malfunctioning; and
based on determining that the first artificial intelligence agent is malfunctioning, transmitting the request to a second artificial intelligence agent.
2 . The one or more non-transitory computer-readable storage media of claim 1 , wherein the instructions further cause the one or more processors to perform operations comprising:
retrieving, using a first agent identifier associated with the first artificial intelligence agent, reputation data associated with the first artificial intelligence agent,
wherein the reputation data comprises response accuracy information for a plurality of responses received from the first artificial intelligence agent; and
inputting the reputation data into a reputation machine learning model to obtain a reputation parameter associated with the first artificial intelligence agent,
wherein the reputation machine learning model has been trained to output a corresponding reputation parameter based on degree of accuracy of past responses and recency of the past responses, and
wherein the reputation parameter is used to determine whether the first artificial intelligence agent is malfunctioning.
3 . The one or more non-transitory computer-readable storage media of claim 2 , wherein the instructions further cause the one or more processors to perform operations comprising:
transmitting the request to the first artificial intelligence agent,
wherein the request comprises a task and a parameter query, and
wherein the parameter query requests the first artificial intelligence agent to respond with one or more estimates for communication load, resource load, and action sequence load for performing the task; and
in response to determining that the communication load, the resource load, or the action sequence load meets a threshold, transmit a cancel request to the first artificial intelligence agent.
4 . The one or more non-transitory computer-readable storage media of claim 1 , wherein the ensemble anomaly detection model:
generates, using an isolation forest model, outlier data that indicates whether one or more outliers are present in the timeseries data associated with the plurality of sets of historical agent parameters of the first artificial intelligence agent,
wherein the isolation forest model is trained to detect outliers within datasets;
generates, using an autoencoder model, reconstruction error data,
wherein the autoencoder model is trained to minimize error values for the historical agent parameters of the first artificial intelligence agent;
generates, using a long term short term memory network, temporal anomaly data indicating whether one or more temporal anomalies exist within the timeseries data associated with the plurality of sets of historical agent parameters of the first artificial intelligence agent;
generates, using a graph neural net, interaction anomaly data indicating whether one or one or more anomalies exist in interaction patterns between the first artificial intelligence agent and other entities;
generates, using a support vector machine model, boundary data indicating whether one or more boundaries exist with the timeseries data associated with the plurality of sets of historical agent parameters of the first artificial intelligence agent,
wherein the one or more boundaries indicate one or more anomalies within the timeseries data;
generates, using a transformer model, attention data indicating whether one or more attention anomalies exist within the historical agent parameters of the first artificial intelligence agent,
wherein the one or more attention anomalies indicate disproportionate focus on a portion of the historical agent parameters of the first artificial intelligence agent; and
determines, using the outlier data, the reconstruction error data, the temporal anomaly data, the interaction anomaly data, the boundary data, and the attention data, that one or more anomalies associated with the first artificial intelligence agent.
5 . The one or more non-transitory computer-readable storage media of claim 1 , wherein the instructions further cause the one or more processors to perform operations comprising:
extracting a cryptographic signature from the response received from the first artificial intelligence agent,
wherein the cryptographic signature has been generated using a private key associated with the first artificial intelligence agent;
retrieving a public key associated with the first artificial intelligence agent;
determining, using the public key associated with the first artificial intelligence agent, whether the cryptographic signature is valid; and
based on determining that the cryptographic signature is invalid, removing the response and transmitting the request to the second artificial intelligence agent.
6 . The one or more non-transitory computer-readable storage media of claim 1 , wherein the instructions further cause the one or more processors to perform operations comprising:
periodically receiving, from the first artificial intelligence agent, a corresponding set of agent parameters comprising corresponding communication parameters, corresponding resource parameters, corresponding decision parameters timing parameters and corresponding application programming interface parameters for the first artificial intelligence agent;
match a first set of agent parameters to processing of the request; and
generate the plurality of agent parameters based on the first set of agent parameters.
7 . The one or more non-transitory computer-readable storage media of claim 1 , wherein the instructions further cause the one or more processors to perform operations comprising:
generating, using historical message frequency data, historical payload size data, and historical response destination data associated with the first artificial intelligence agent, a communication profile for the first artificial intelligence agent,
wherein the communication profile is generated by aggregating the historical message frequency data, the historical payload size data, and the historical response destination data; and
generating, using historical processor usage data, historical memory pattern data, and historical input-output data, the resource profile for the first artificial intelligence agent,
wherein the resource profile is generated by aggregating the historical processor usage data, the historical memory pattern data, and the historical input-output data.
8 . The one or more non-transitory computer-readable storage media of claim 7 , wherein the instructions further cause the one or more processors to perform operations comprising:
generating, for the first artificial intelligence agent using vote data accuracy data and timing data, a decision profile for the first artificial intelligence agent;
determining, for the first artificial intelligence agent, a plurality of temporal cycles, a plurality of bursts, and a plurality of trends based on historical data generated when processing a plurality of requests;
adding the plurality of temporal cycles, the plurality of bursts, and the plurality of trends into a temporal profile; and
generating the agent profile by aggregating the communication profile, the resource profile, the decision profile and the temporal profile.
9 . A method for detecting malfunctions in artificial intelligence agents, the method comprising:
receiving, from a first artificial intelligence agent of a plurality of artificial intelligence agents, a response to a request,
wherein each artificial intelligence agent of the plurality of artificial intelligence agents is associated with a corresponding computer-executable operation set configured to be autonomously executed by a corresponding artificial intelligence agent on a software application set;
retrieving (1) a plurality of agent parameters associated with the first artificial intelligence agent, (2) an agent profile associated with the first artificial intelligence agent, and (3) a plurality of sets of historical agent parameters,
wherein the plurality of agent parameters comprises one or more of communication parameters, resource parameters or timing parameters, and
wherein the agent profile comprises one or more of a communications profile, a resource profile, or a timing profile;
inputting the plurality of agent parameters and the agent profile into a machine learning model to obtain a behavior parameter that indicates whether the plurality of agent parameters match the agent profile,
wherein the machine learning model is trained, using historical communication data, historical resource data, and historical timing data associated with the first artificial intelligence agent, to recognize patterns within the plurality of agent parameters to match with corresponding patterns within the agent profile;
inputting the plurality of agent parameters into an ensemble anomaly detection model to obtain an anomaly parameter whether the plurality of agent parameters indicate an anomaly with the response,
wherein the ensemble anomaly detection model is trained to identify anomalies between the plurality of agent parameters and timeseries data associated with the plurality of sets of historical agent parameters;
determining based on the behavior parameter and the anomaly parameter whether the first artificial intelligence agent is malfunctioning; and
based on determining that the first artificial intelligence agent is malfunctioning, transmitting the request to a second artificial intelligence agent.
10 . The method of claim 9 , further comprising:
retrieving, using a first agent identifier associated with the first artificial intelligence agent, reputation data associated with the first artificial intelligence agent,
wherein the reputation data comprises response accuracy information for a plurality of responses received from the first artificial intelligence agent; and
inputting the reputation data into a reputation machine learning model to obtain a reputation parameter associated with the first artificial intelligence agent,
wherein the reputation machine learning model has been trained to output a corresponding reputation parameter based on degree of accuracy of past responses and recency of the past responses, and
wherein the reputation parameter is used to determine whether the first artificial intelligence agent is malfunctioning.
11 . The method of claim 10 , further comprising:
transmitting the request to the first artificial intelligence agent,
wherein the request comprises a task and a parameter query, and
wherein the parameter query requests the first artificial intelligence agent to respond with one or more estimates for communication load, resource load, and action sequence load for performing the task; and
in response to determining that the communication load, the resource load, or the action sequence load meets a threshold, transmit a cancel request to the first artificial intelligence agent.
12 . The method of claim 9 , wherein the ensemble anomaly detection model:
generates, using an isolation forest model, outlier data that indicates whether one or more outliers are present in the timeseries data associated with the plurality of sets of historical agent parameters of the first artificial intelligence agent,
wherein the isolation forest model is trained to detect outliers within datasets;
generates, using an autoencoder model, reconstruction error data,
wherein the autoencoder model is trained to minimize error values for the historical agent parameters of the first artificial intelligence agent;
generates, using a long term short term memory network, temporal anomaly data indicating whether one or more temporal anomalies exist within the timeseries data associated with the plurality of sets of historical agent parameters of the first artificial intelligence agent;
generates, using a graph neural net, interaction anomaly data indicating whether one or one or more anomalies exist in interaction patterns between the first artificial intelligence agent and other entities;
generates, using a support vector machine model, boundary data indicating whether one or more boundaries exist with the timeseries data associated with the plurality of sets of historical agent parameters of the first artificial intelligence agent,
wherein the one or more boundaries indicate one or more anomalies within the timeseries data;
generates, using a transformer model, attention data indicating whether one or more attention anomalies exist within the historical agent parameters of the first artificial intelligence agent,
wherein the one or more attention anomalies indicate disproportionate focus on a portion of the historical agent parameters of the first artificial intelligence agent; and
determines, using the outlier data, the reconstruction error data, the temporal anomaly data, the interaction anomaly data, the boundary data, and the attention data, that one or more anomalies associated with the first artificial intelligence agent.
13 . The method of claim 9 , further comprising:
extracting a cryptographic signature from the response received from the first artificial intelligence agent,
wherein the cryptographic signature has been generated using a private key associated with the first artificial intelligence agent;
retrieving a public key associated with the first artificial intelligence agent;
determining, using the public key associated with the first artificial intelligence agent, whether the cryptographic signature is valid; and
based on determining that the cryptographic signature is invalid, removing the response and transmit the request to the second artificial intelligence agent.
14 . The method of claim 9 , further comprising:
periodically receiving, from the first artificial intelligence agent, a corresponding set of agent parameters comprising corresponding communication parameters, corresponding resource parameters, corresponding decision parameters timing parameters and corresponding application programming interface parameters for the first artificial intelligence agent;
match a first set of agent parameters to processing of the request; and
generate the plurality of agent parameters based on the first set of agent parameters.
15 . The method of claim 9 , further comprising:
generating, using historical message frequency data, historical payload size data, and historical response destination data associated with the first artificial intelligence agent, a communication profile for the first artificial intelligence agent,
wherein the communication profile is generated by aggregating the historical message frequency data, the historical payload size data, and the historical response destination data; and
generating, using historical processor usage data, historical memory pattern data, and historical input-output data, the resource profile for the first artificial intelligence agent,
wherein the resource profile is generated by aggregating the historical processor usage data, the historical memory pattern data, and the historical input-output data.
16 . The method of claim 15 , further comprising:
generating, for the first artificial intelligence agent using vote data accuracy data and timing data, a decision profile for the first artificial intelligence agent;
determining, for the first artificial intelligence agent, a plurality of temporal cycles, a plurality of bursts, and a plurality of trends based on historical data generated when processing a plurality of requests;
adding the plurality of temporal cycles, the plurality of bursts, and the plurality of trends into a temporal profile; and
generating the agent profile by aggregating the communication profile, the resource profile, the decision profile and the temporal profile.
17 . A system for detecting malfunctions in artificial intelligence agents, the system comprising:
one or more processors; and
one or more memories storing instructions, which when executed by the one or more processors cause the one or more processors to perform operations comprising:
receiving, from a first artificial intelligence agent of a plurality of artificial intelligence agents, a response to a request,
wherein each artificial intelligence agent of the plurality of artificial intelligence agents is associated with a corresponding computer-executable operation set configured to be autonomously executed by a corresponding artificial intelligence agent on a software application set;
retrieving (1) a plurality of agent parameters associated with the first artificial intelligence agent, (2) an agent profile associated with the first artificial intelligence agent, and (3) a plurality of sets of historical agent parameters,
wherein the plurality of agent parameters comprises one or more of communication parameters, resource parameters or timing parameters, and
wherein the agent profile comprises one or more of a communications profile, a resource profile, or a timing profile;
inputting the plurality of agent parameters and the agent profile into a machine learning model to obtain a behavior parameter that indicates whether the plurality of agent parameters match the agent profile,
wherein the machine learning model is trained, using historical communication data, historical resource data, and historical timing data associated with the first artificial intelligence agent, to recognize patterns within the plurality of agent parameters to match with corresponding patterns within the agent profile;
inputting the plurality of agent parameters into an ensemble anomaly detection model to obtain an anomaly parameter whether the plurality of agent parameters indicate an anomaly with the response,
wherein the ensemble anomaly detection model is trained to identify anomalies between the plurality of agent parameters and timeseries data associated with the plurality of sets of historical agent parameters;
determining based on the behavior parameter and the anomaly parameter whether the first artificial intelligence agent is malfunctioning; and
based on determining that the first artificial intelligence agent is malfunctioning, transmitting the request to a second artificial intelligence agent.
18 . The system of claim 17 , wherein the instructions further cause the one or more processors to perform operations comprising:
retrieving, using a first agent identifier associated with the first artificial intelligence agent, reputation data associated with the first artificial intelligence agent,
wherein the reputation data comprises response accuracy information for a plurality of responses received from the first artificial intelligence agent; and
inputting the reputation data into a reputation machine learning model to obtain a reputation parameter associated with the first artificial intelligence agent,
wherein the reputation machine learning model has been trained to output a corresponding reputation parameter based on degree of accuracy of past responses and recency of the past responses, and
wherein the reputation parameter is used to determine whether the first artificial intelligence agent is malfunctioning.
19 . The system of claim 18 , wherein the instructions further cause the one or more processors to perform operations comprising:
transmitting the request to the first artificial intelligence agent,
wherein the request comprises a task and a parameter query, and
wherein the parameter query requests the first artificial intelligence agent to respond with one or more estimates for communication load, resource load, and action sequence load for performing the task; and
in response to determining that the communication load, the resource load, or the action sequence load meets a threshold, transmit a cancel request to the first artificial intelligence agent.
20 . The system of claim 17 , wherein the instructions further cause the one or more processors to perform operations comprising:
extracting a cryptographic signature from the response received from the first artificial intelligence agent, wherein the cryptographic signature has been generated using a private key associated with the first artificial intelligence agent;
retrieving a public key associated with the first artificial intelligence agent;
determining, using the public key associated with the first artificial intelligence agent, whether the cryptographic signature is valid; and
based on determining that the cryptographic signature is invalid, removing the response and transmit the request to the second artificial intelligence agent.