Autonomous agent observation and control
Systems, methods, and devices that relate to monitoring and managing autonomous agents are disclosed. In one example aspect, the method includes receiving activity data from autonomous agents in an operational environment, deploying static and dynamic observing agents to monitor expected behavior and deviations, detecting a deviation by an autonomous agent, determining the cause through analysis, performing a mitigative action based on the cause, and executing a preventative action to block similar future deviations. The method may also involve configuring observing agents with different observation modalities, periodically modifying observation parameters unpredictably, facilitating direct communication between observing agents, resolving conflicts in observations, and updating observation policies. Mitigative actions can include disabling credentials, rerouting communications, and logging actions. Preventative measures may involve updating behavioral policies and adjusting agent parameters to disincentivize problematic behaviors.
1 . One or more non-transitory, computer-readable storage medium comprising instructions recorded thereon, wherein the instructions, when executed by at least one data processor of a system, cause the system to:
receive, from a plurality of autonomous agents deployed within an operational environment, a plurality of activity data streams;
deploy a plurality of observing agents for monitoring the plurality of autonomous agents;
detect, using at least one observing agent of the plurality of observing agents, a deviation of an autonomous agent of the plurality of autonomous agents from expected agent behavior;
identify a decision path responsible for the deviation, wherein the decision path is associated with the autonomous agent;
execute, based on the decision path, a mitigative action comprising:
generating a revocation signal that disables at least one credential, token, or network route previously assigned to the autonomous agent associated with the decision path; and
rerouting all subsequent communications originating from or directed to the autonomous agent; and
store the mitigative action, a timestamp, and the decision path, on an auditable ledger.
2 . The one or more non-transitory, computer-readable storage medium of claim 1 , wherein the plurality of observing agents comprises at least one dynamic observing agent configured to detect deviations from the expected agent behavior, and wherein the instructions further cause the system to:
configure each observing agent of the plurality of observing agents with a respective observation modality selected from a plurality of observation modalities, wherein at least two observing agents of the plurality of observing agents are configured with different observation modalities; and
periodically perform, for the at least one dynamic observing agent of the plurality of observing agents, modifications to an observation parameter set comprising at least one of sampling frequency, data-collection granularity, vantage point within the operational environment, and obfuscation level, such that each modification is unpredictable to the plurality of autonomous agents.
3 . The one or more non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:
cause a first observing agent of the plurality of observing agents to transmit a first observation summary directly to a second observing agent of the plurality of observing agents, wherein the first observing agent is a static observing agent and the second observing agent is a dynamic observing agent;
compare, at the second observing agent, the first observation summary generated by the first observing agent with a second observation summary generated by the second observing agent to detect a conflict between the first observation summary and the second observation summary, the conflict regarding at least one of the plurality of autonomous agents;
based on detecting the conflict, apply, at the second observing agent, a conflict-resolution rule set comprising a precedence rule and a tie-breaking rule to generate a resolved determination for the conflict; and
update an observation policy for the plurality of observing agents based on the resolved determination.
4 . The one or more non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to execute a preventative action to block future decision paths similar to the decision path by the autonomous agent or by other autonomous agents of the plurality of autonomous agents, wherein the preventative action is based on the deviation and the mitigative action, and wherein the instructions for executing the preventative action further cause the system to:
update a behavioral policy to include a rule that blocks the decision path and similar decision paths matching the decision path within a similarity threshold;
adjust a parameter of the autonomous agent to disincentivize future selection of the decision path or the similar decision paths; and
transmit the updated behavioral policy to the plurality of autonomous agents.
5 . The one or more non-transitory, computer-readable storage medium of claim 1 , wherein the plurality of observing agents comprises at least one static observing agent configured to monitor for the expected agent behavior and at least one dynamic observing agent configured to detect deviations from the expected agent behavior, and wherein the instructions for detecting the deviation further cause the system to:
employ the at least one dynamic observing agent to perform anomaly detection on the plurality of activity data streams by comparing current activity metrics to a learned baseline to identify unexpected actions or inactions;
employ the at least one static observing agent to compare the unexpected actions and the inactions of the autonomous agent to a predefined set of expected behavior patterns; and
detect the deviation based on (i) the at least one dynamic observing agent detecting an unexpected action or an unexpected inaction by the autonomous agent and (ii) the at least one static observing agent failing to identify a corresponding expected behavior pattern from the predefined set of expected behavior patterns.
6 . A method comprising:
receiving, from a plurality of autonomous agents operating within an operational environment, activity data;
deploying a plurality of observing agents for monitoring the plurality of autonomous agents;
detecting, by at least one of the plurality of observing agents, a deviation of an autonomous agent from an expected behavior pattern;
determining, based on an analysis of the activity data, a cause of the deviation; and
performing a mitigative action with respect to the autonomous agent based on the cause of the deviation, the mitigative action comprising:
generating a revocation signal that disables at least one credential, token, or network route previously assigned to the autonomous agent associated with the cause of the deviation; and
rerouting all subsequent communications originating from or directed to the autonomous agent; and
storing the mitigative action, a timestamp, and the cause, on an auditable ledger.
7 . The method of claim 6 , wherein the plurality of observing agents comprises at least one dynamic observing agent configured to detect deviations from the expected behavior pattern, further comprising:
configuring each observing agent of the plurality of observing agents with a respective observation modality selected from a plurality of observation modalities, wherein at least two observing agents of the plurality of observing agents are configured with different observation modalities; and
periodically performing, for the at least one dynamic observing agent of the plurality of observing agents, modifications to an observation parameter set comprising at least one of sampling frequency, data-collection granularity, vantage point within the operational environment, and obfuscation level, such that each modification is unpredictable to the plurality of autonomous agents.
8 . The method of claim 6 , further comprising:
causing a first observing agent of the plurality of observing agents to transmit a first observation summary directly to a second observing agent of the plurality of observing agents, wherein the first observing agent is a static observing agent and the second observing agent is a dynamic observing agent;
comparing, at the second observing agent, the first observation summary generated by the first observing agent with a second observation summary generated by the second observing agent to detect a conflict between the first observation summary and the second observation summary, the conflict regarding at least one of the plurality of autonomous agents;
based on detecting the conflict, applying, at the second observing agent, a conflict-resolution rule set comprising a precedence rule and a tie-breaking rule to generate a resolved determination for the conflict; and
updating an observation policy for the plurality of observing agents based on the resolved determination.
9 . The method of claim 6 , further comprising performing a preventative action to block future deviations similar to the deviation by the autonomous agent or by other autonomous agents of the plurality of autonomous agents, wherein the preventative action is based on the deviation, the analysis, and the mitigative action, and wherein performing the preventative action further comprises:
updating a behavioral policy to include a rule that blocks the cause and similar causes matching the cause within a similarity threshold;
adjusting a parameter of the autonomous agent to disincentivize future selection of the cause or the similar causes; and
transmitting the updated behavioral policy to the plurality of autonomous agents.
10 . The method of claim 6 , wherein the plurality of observing agents comprises at least one static observing agent configured to monitor for the expected behavior pattern and at least one dynamic observing agent configured to detect deviations from the expected behavior pattern, and wherein detecting the deviation further comprises:
employing the at least one dynamic observing agent to perform anomaly detection on the activity data by comparing current activity metrics to a learned baseline to identify unexpected actions or inactions;
employing the at least one static observing agent to compare actions and inactions of the autonomous agent to a predefined set of expected behavior patterns; and
detecting the deviation based on (i) the at least one dynamic observing agent detecting an unexpected action or an unexpected inaction by the autonomous agent and (ii) the at least one static observing agent failing to identify a corresponding expected behavior pattern from the predefined set of expected behavior patterns.
11 . The method of claim 6 , wherein the mitigative action comprises at least one of interrupting execution of the autonomous agent, suspending the autonomous agent, shutting down the autonomous agent, or isolating a subsystem of the autonomous agent.
12 . A system comprising:
a storage device; and
one or more processors communicatively coupled to the storage device storing instructions thereon, that cause the one or more processors to:
receive, from a plurality of autonomous agents operating within an operational environment, activity data;
deploy a plurality of observing agents for monitoring the plurality of autonomous agents;
detect, by at least one of the plurality of observing agents, a deviation of an autonomous agent from an expected behavior pattern;
determine, based on an analysis of the activity data, a factor contributing to the deviation; and
perform a mitigative action with respect to the autonomous agent based on the factor contributing to the deviation, the mitigative action comprising:
generating a revocation signal that disables at least one credential, token, or network route previously assigned to the autonomous agent associated with the factor contributing to the deviation; and
rerouting all subsequent communications originating from or directed to the autonomous agent; and
store the mitigative action, a timestamp, and the factor, on an auditable ledger.
13 . The system of claim 12 , wherein the plurality of observing agents comprises at least one dynamic observing agent configured to detect deviations from the expected behavior pattern, and wherein the instructions further cause the one or more processors to:
configure each observing agent of the plurality of observing agents with a respective observation modality selected from a plurality of observation modalities, wherein at least two observing agents of the plurality of observing agents are configured with different observation modalities; and
periodically perform, for the at least one dynamic observing agent of the plurality of observing agents, modifications to an observation parameter set comprising at least one of sampling frequency, data-collection granularity, vantage point within the operational environment, and obfuscation level, such that each modification is unpredictable to the plurality of autonomous agents.
14 . The system of claim 12 , wherein the instructions further cause the one or more processors to:
cause a first observing agent of the plurality of observing agents to transmit a first observation summary directly to a second observing agent of the plurality of observing agents, wherein the first observing agent is a static observing agent and the second observing agent is a dynamic observing agent;
compare, at the second observing agent, the first observation summary generated by the first observing agent with a second observation summary generated by the second observing agent to detect a conflict between the first observation summary and the second observation summary, the conflict regarding at least one of the plurality of autonomous agents;
based on detecting the conflict, apply, at the second observing agent, a conflict-resolution rule set comprising a precedence rule and a tie-breaking rule to generate a resolved determination for the conflict; and
update an observation policy for the plurality of observing agents based on the resolved determination.
15 . The system of claim 12 , wherein the instructions further cause the one or more processors to perform a preventative action to block future deviations similar to the deviation by the autonomous agent or by other autonomous agents of the plurality of autonomous agents, wherein the preventative action is based on the deviation, the analysis, and the mitigative action, and wherein the instructions for executing the preventative action further cause the one or more processors to:
update a behavioral policy to include a rule that blocks the factor and similar factors matching the factor within a similarity threshold;
adjust a parameter of the autonomous agent to disincentivize future selection of the factor or the similar factors; and
transmit the updated behavioral policy to the plurality of autonomous agents.
16 . The system of claim 12 , wherein the plurality of observing agents comprises at least one static observing agent configured to monitor for the expected behavior pattern and at least one dynamic observing agent configured to detect deviations from the expected behavior pattern, and wherein the instructions for detecting the deviation further cause the one or more processors to:
employ the at least one dynamic observing agent to perform anomaly detection on the activity data by comparing current activity metrics to a learned baseline to identify unexpected actions or inactions;
employ the at least one static observing agent to compare actions and inactions of the autonomous agent to a predefined set of expected behavior patterns; and
detect the deviation based on (i) the at least one dynamic observing agent detecting an unexpected action or an unexpected inaction by the autonomous agent and (ii) the at least one static observing agent failing to identify a corresponding expected behavior pattern from the predefined set of expected behavior patterns.
17 . The system of claim 12 , wherein the mitigative action comprises at least one of interrupting execution of the autonomous agent, suspending the autonomous agent, shutting down the autonomous agent, or isolating a subsystem of the autonomous agent.