Behavioral analysis of autonomous artificial intelligence (AI) agents deployed in real-world systems

1 . A method of identifying and outputting data characterizing an explanation of behavior exhibited by an autonomous AI agent in such a way that behavior correction can be dynamically implemented, the method comprising:

providing an autonomous agent that when deployed as executable code in a real-world system to control hardware and/or software or in a computer-implemented training platform, the autonomous AI agent exhibits behavior by perceiving the environment, taking actions autonomously to achieve goals, and improving agent performance through learning;

tracking behavior iterations of the autonomous AI agent reacting with an environment during a training or run cycle of the autonomous AI;

collecting iteration data from the tracked iterations, the iteration data comprising which components of the autonomous AI agent are used for each iteration;

transforming the iteration data into the data characterizing the explanation of the behavior exhibited by the autonomous AI agent at different levels of granularity of the training or run cycle, the transforming comprising:

identifying data characterizing the behavior exhibited by the autonomous AI agent for each individual iteration of a training or run cycle;

identifying data characterizing the behavior exhibited by the autonomous AI agent grouped by episodes of the training or run cycle, wherein each episode represents multiple iterations which are a subset of all iterations of an entirety of the training or run cycle; and

identifying data characterizing the behavior exhibited by the autonomous AI agent across an entirety of the training or run cycle comprising all the iterations; and

outputting the data characterizing the explanation of behavior exhibited by the autonomous AI agent that indicates what, how, why, and/or when for the behavior of the autonomous AI agent during the training or run cycle, enabling dynamic behavior correction for the autonomous AI agent.

2 . The method of claim 1 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent for an individual iteration comprises a fingerprint indicating what components of the autonomous AI agent were utilized during the iteration.

3 . The method of claim 1 , wherein each episode corresponds to behavior by the autonomous AI agent that is identified as significant/interesting based on a score awarded for the behavior.

4 . The method of claim 3 , wherein behavior is identified as significant/interesting based on goals for the autonomous AI agent.

5 . The method of claim 1 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent grouped by episodes comprises a graphical timeline of episodes.

6 . The method of claim 1 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent grouped by episodes comprises an indication of a status of the environment and how the autonomous AI agent reacted at a time of each episode.

7 . The method of claim 1 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent across the entirety of the training or run cycle comprises a combination and clustering of data from the identified data characterizing the behavior exhibited by the autonomous AI agent for each individual iteration of the entirety of the training or run cycle.

8 . The method of claim 7 , wherein the combination and clustering are performed using a machine learning algorithm.

9 . The method of claim 7 , wherein resulting clusters represent states of the autonomous AI agent across the entirety of the training or run cycle.

10 . The method of claim 1 , wherein the autonomous AI agent comprises:

one or more sensor modules;

optionally one or more perceptor modules;

one or more scenario modules;

one or more skills modules; and

one or more performable actions.

11 . The method of claim 10 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior for each individual iteration of a training or run cycle comprises a graphical indication of which of: the one or more sensor modules, the optional one or more perceptor modules, the one or more scenario modules, the one or more skills modules, and the one or performable actions of the autonomous AI agent were used in the iteration.

12 . The method of claim 1 , wherein the tracking, collecting, transforming, and outputting steps are performed by the autonomous AI agent.

13 . The method of claim 1 , wherein the method is performed by a platform for creating autonomous AI agents.

14 . The method of claim 1 , wherein outputting the data characterizing the explanation of behavior exhibited by the autonomous AI agent comprises:

outputting one or more of:

identified data characterizing the behavior exhibited by the autonomous AI agent for an individual iteration of a training or run cycle;

identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior grouped by episodes of the training or run cycle; and

identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior across the entirety of the training or run cycle.

15 . The method of claim 14 , wherein identified data characterizing the behavior exhibited by the autonomous AI agent for an individual iteration of a training or run cycle is outputted in response to selection on an outputted identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior grouped by episodes of the training or run cycle or an outputted identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior across the entirety of training or run cycle.

16 . A system for identifying and outputting data characterizing an explanation of behavior exhibited by an autonomous AI agent in such a way that enables dynamic behavior correction, the system comprising:

an autonomous agent that when deployed as executable code in a real-world system to control hardware and/or software or in a computer-implemented training platform, the autonomous AI agent exhibits behavior by perceiving the environment, taking actions autonomously to achieve goals, and improving agent performance through learning;

a storage holding data; and

a processor in communication with the storage, wherein the processor:

tracks behavior iterations of the autonomous AI agent reacting with an environment during a training or run cycle of the autonomous AI agent;

collects iteration data from the tracked iterations, the iteration data comprising which components of the autonomous AI agent are used for each iteration;

transforms the iteration data into the data characterizing the explanation of the behavior exhibited by the autonomous AI agent at different levels of granularity of the training or run cycle, the transforming comprising:

identifying data characterizing the behavior exhibited by the autonomous AI agent for each individual iteration of a training or run cycle;

identifying data characterizing the behavior exhibited by the autonomous AI agent grouped by episodes of the training or run cycle, wherein each episode represents multiple iterations that are a subset of all iterations of an entirety of the training or run cycle; and

identifying data characterizing the behavior exhibited by the autonomous AI agent across an entirety of the training or run cycle comprising all the iterations; and

outputs the data characterizing the explanation of behavior exhibited by the autonomous AI agent that indicates what, how, why, and/or when for the behavior of the autonomous AI agent during the training or run cycle, enabling dynamic behavior correction.

17 . The system of claim 16 further comprising:

a display in communication with the processor configured to display the data characterizing the explanation of behavior exhibited by the autonomous AI agent output by the processor.

18 . The system of claim 16 , wherein the processor is also executing the autonomous AI agent.

19 . The system of claim 16 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent for an individual iteration comprises a fingerprint indicating what components of the autonomous AI agent were utilized during the iteration.

20 . The system of claim 16 , wherein each episode corresponds to behavior by the autonomous AI agent that is identified as significant/interesting based on a score awarded for the behavior.

21 . The system of claim 20 , wherein behavior is identified as significant/interesting based on goals for the autonomous AI agent.

22 . The system of claim 16 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent grouped by episodes comprises a graphical timeline of episodes.

23 . The system of claim 16 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent grouped by episodes comprises an indication of a status of the environment and how the autonomous AI agent reacted at a time of each episode.

24 . The system of claim 16 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent across the entirety of the training or run cycle comprises a combination and clustering of data from the identified data characterizing the behavior exhibited by the autonomous AI agent for each individual iteration of the entirety of the training or run cycle.

25 . The system of claim 24 , wherein the combination and clustering are performed using a machine learning algorithm.

26 . The system of claim 24 , wherein resulting clusters represent states of the autonomous AI agent across the entirety of the training or run cycle.

27 . The system of claim 16 , wherein the autonomous AI agent comprises:

one or more sensor modules,

optionally one or more perceptor modules,

one or more scenario modules,

one or more skills modules, and

one or more performable actions.

28 . The system of claim 27 , wherein the identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior for each individual iteration of a training or run cycle comprises a graphical indication of which of: the one or more sensor modules, the optional one or more perceptor modules, the one or more scenario modules, the one or more skills modules, and the one or performable actions of the autonomous AI agent were used in the iteration.

29 . The system of claim 16 , wherein outputting the data characterizing the explanation of behavior exhibited by the autonomous AI agent comprises:

outputting one or more of:

identified data characterizing the behavior exhibited by the autonomous AI agent for an individual iteration of a training or run cycle;

identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior grouped by episodes of the training or run cycle; and

identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior across the entirety of the training or run cycle.

30 . The system of claim 29 , wherein identified data characterizing the behavior exhibited by the autonomous AI agent for an individual iteration of a training or run cycle is outputted in response to selection on an outputted identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior grouped by episodes of the training or run cycle or an outputted identified data characterizing the behavior exhibited by the autonomous AI agent of the behavior across the entirety of training or run cycle.