Behaviour modeling, verification, and autonomous actions and triggers of ML and AI systems
An exemplary embodiment may present a behavior modeling architecture that is intended to assist in handling, modelling, predicting and verifying the behavior of machine learning models to assure the safety of such systems meets the required specifications and adapt such architecture according to the execution sequences of the behavioral model. An embodiment may enable conditions in a behavioral model to be integrated in the execution sequence of behavioral modeling in order to monitor the probability likelihoods of certain paths in a system. An embodiment allows for real-time monitoring during training and prediction of machine learning models. Conditions may also be utilized to trigger system-knowledge injection in a white-box model in order to maintain the behavior of a system within defined boundaries. An embodiment further enables additional formal verification constraints to be set on the output or internal parts of white-box models.
1 . A computer-implemented behavioral modeling method for handling, modeling, predicting, and verifying a behavior of a system comprising an explainable architecture configured to provide at least one explainable model, comprising executing, on a processor and via a behavioral modeling architecture of the system separate from the explainable architecture, the steps of:
integrating at least one condition into at least one of a plurality of execution sequences in the behavioral modeling architecture, each condition based on one of an internal coefficient of the system or an environmental variable of the system, each condition configured to trigger at least one action if activated, and determining, for each of the plurality of execution sequences, a probability likelihood of encountering each of the plurality of execution sequences;
identifying at least one explanation corresponding to the at least one action in an action space, said explanation associated with at least one state and at least one reward, wherein at least one of the action space and the state space is discrete or continuous;
predicting and storing at least one predicted boundary associated with the at least one explainable model;
operating the system comprising the at least one explainable model, and continuously monitoring at least one variable including at least one of: one or more internal coefficients of the system in a system memory and one or more environmental variables in a physical space, wherein continuously monitoring comprises comparing the at least one variable to the at least one condition to determine whether the at least one condition has been met;
determining, based on monitoring the at least one condition, that the at least one condition has been met and that the at least one action paired with the at least one condition has been triggered, wherein the condition is triggered by data received from the explainable model via the behavioral modeling architecture separate from the explainable architecture;
determining an actual boundary demonstrated by triggering of the at least one condition, and, based on observation of the actual boundary, identifying a gap between the predicted boundary of the at least one explainable model and the actual boundary;
updating the at least one predicted boundary of the at least one explainable model based on information observed during monitoring of the at least one condition and based on the gap identified between the predicted boundary of the at least one explainable model and the actual boundary.
2 . The computer implemented behavioral modeling system according to claim 1 , wherein the computer implemented behavioral modeling system comprises one or more sensors, wherein the predefined actual boundary is based on an output of the one or more sensors.
3 . The computer implemented behavioral modeling system according to claim 2 , wherein the system is an autonomous vehicle system, and wherein the one or more sensors comprise one or more of: a plurality of camera systems providing coverage of incoming objects from multiple directions of the autonomous vehicle system, one or more hyper-spectral cameras configured to provide hyper-spectral data, one or more audio sensors configured to provide audio data, one or more haptic sensors configured to provide haptic data, one or more Light Detection and Ranging (LIDAR) sensors configured to provide LIDAR data, one or more radar sensors configured to provide radar data, and one or more sonar sensors configured to provide sonar data.
4 . The computer implemented behavioral modeling system according to claim 3 , wherein identifying the gap between the predicted boundary of the system and the predefined actual boundary of the system comprises adjusting an observation priority for a first set of incoming objects provided in a first location, relative to a second set of incoming objects provided in a second location.
5 . The computer implemented behavioral modeling system according to claim 3 , wherein the one or more instances of tuning the behavioral modeling architecture comprises creation of one or more constraints or rules for navigation of the autonomous vehicle system.
6 . The computer implemented behavioral modeling system according to claim 5 , wherein creation of the one or more constraints or rules further comprises generation and output of one or more explanations for the one or more constraints or rules.
7 . The computer implemented behavioral modeling system according to claim 6 , wherein the predefined actual boundary based on the output of the one or more sensors is based on at least one of: an output of an acceleration sensor configured to detect at least one of an acceleration magnitude and an acceleration direction, and an output of an image recognition sensor configured to identify one or more objects.
8 . The computer implemented behavioral modeling system according to claim 6 , wherein generation and output of the one or more explanations comprises outputting the one or more explanations to at least one of: a dashboard display of the autonomous vehicle system, a heads-up display of the autonomous vehicle system, and an augmented reality display of the autonomous vehicle system.
9 . The computer implemented behavioral modeling system according to claim 6 , wherein generation and output of the one or more explanations comprises transmitting the one or more explanations to at least one other vehicle via an Intelligent Transport System (ITS) network, the ITS network comprising one of a vehicle to vehicle (V2V) network, a vehicular ad hoc network (VANET), or a mobile ad hoc network (MANET).
10 . The computer implemented behavioral modeling system according to claim 1 , further comprising a causal model separate from the system and configured to provide at least one of: one or more triggers, one or more events, and one or more actions, wherein the at least one of the one or more triggers, the one or more events, and the one or more actions correspond to the conditions integrated into the behavioral modeling architecture.
11 . The computer implemented behavioral modeling system according to claim 10 , wherein identifying the gap between the predicted boundary of the system and the predefined actual boundary of the system comprises measuring or estimating effects of a first cause of the causal model and of a second cause of the causal model, and comparing relative effects of the first cause and the second cause.
12 . The computer implemented behavioral modeling system according to claim 11 , wherein measuring or estimating the effects of the first cause of the causal model and of the second cause of the causal model comprises measuring or estimating relative effects of a first medical cause on a patient and a second medical cause on the patient.
13 . The computer implemented behavioral modeling system according to claim 1 , wherein the system is a distributed explainable architecture (DEA) comprising a plurality of explainable models configured in parallel, each explainable model comprising one of a neural network, an explainable artificial intelligence model, or a machine learning algorithm.
14 . The computer implemented behavioral modeling system according to claim 13 , wherein the DEA is a hybrid of different types of explainable models, the plurality of explainable models comprising at least two of the neural network, the explainable artificial intelligence model, and the machine learning algorithm, wherein each of the different types of explainable models is configured independently.
15 . The computer implemented behavioral modeling system according to claim 13 , wherein the behavioral modeling architecture comprises a plurality of triggers corresponding to a plurality of conditions to be activated, a first trigger in the plurality of triggers corresponding to a first action of a first explainable model in the plurality of explainable models and a second trigger in the plurality of triggers corresponding to a second action of a second explainable model in the plurality of explainable models.
16 . The computer implemented behavioral modeling system according to claim 1 , wherein the computer implemented behavioral modeling system is configured to receive, from a user interface, at least one of: a designation of one or more named reference labels associated with the system or with the behavioral modeling architecture, or a deletion of the one or more named reference labels.
17 . The computer implemented behavioral modeling system according to claim 16 , wherein the computer implemented behavioral modeling system is configured to, upon designation of the one or more named reference labels associated with the system or with the behavioral modeling architecture, setting an element of the system or the behavioral modeling architecture that is associated with the one or more named reference labels as invariant, and restricting alteration or deletion of the element.
18 . The computer implemented behavioral modeling system according to claim 1 , wherein the system includes at least one quantum logic specific operator or quantum hardware logic gate, wherein the behavioral modeling architecture is a quantum behavioral modeling architecture, and wherein the predicted boundary of the system is based on a plurality of predicted outcomes and the predefined actual boundary of the system is based on a plurality of outcomes.
19 . A computer implemented behavioral modeling method for handling, modeling, predicting, and verifying a behavior of a system comprising an explainable architecture configured to provide at least one of a neural network, an explainable artificial intelligence model, or a machine learning algorithm, comprising executing, on a processor and via a behavioral modeling architecture of the system separate from the explainable architecture, steps of:
predicting and storing, with the processor, a predicted boundary of the system;
integrating a plurality of conditions into an activation process of the behavioral modeling architecture, each condition based on one of an internal coefficient of the system or an environmental variable of the system, and, during training and prediction of the system, monitoring the plurality of conditions to determine whether any of the plurality of conditions are met;
operating the system, and continuously monitoring at least one variable including at least one of: one or more internal coefficients of the system in a system memory and one or more environmental variables in a physical space, wherein continuously monitoring comprises comparing the at least one variable to the at least one condition to determine whether the at least one condition has been met;
upon determination that one or more of the integrated conditions has been met, based on triggering of the one or more integrated conditions by data received from the at least one of the neural network, explainable artificial intelligence model, or machine learning algorithm via the behavioral modeling architecture separate from the explainable architecture, triggering a system-knowledge injection configured to maintain the behavior of the system within a predefined actual boundary;
identifying a gap between the predicted boundary of the system and the predefined actual boundary of the system;
in a case where the predicted boundary of the system extends outside of the predefined actual boundary, identifying the gap as an underspecified gap, and in a case where the predicted boundary of the system is inside of the predefined actual boundary, identifying the gap as a non-optimal gap;
performing one or more instances of tuning the behavioral modeling architecture, wherein tuning the behavioral modeling architecture comprises updating the predicted boundary based on information observed during the monitoring; and
finalizing the behavioral modeling architecture by terminating tuning of the behavioral modeling architecture upon determination that the gap between the predicted boundary of the system and the predefined actual boundary of the system is reduced.
20 . A non-transitory computer-readable medium comprising program code that, when executed by a processor, is configured to configure the processor to perform a process of handling, modeling, predicting, and verifying a behavior of a system comprising an explainable architecture configured to provide at least one of a neural network, an explainable artificial intelligence model, or a machine learning algorithm, wherein the process comprises performing, with the processor and via a behavioral modeling architecture of the system separate from the explainable architecture, steps of:
predicting and storing, with the processor, a predicted boundary of the system;
integrating a plurality of conditions into an activation process of the behavioral modeling architecture, each condition based on one of an internal coefficient of the system or an environmental variable of the system, and, during training and prediction of the system, monitoring the plurality of conditions to determine whether any of the plurality of conditions are met;
operating the system, and continuously monitoring at least one variable including at least one of: one or more internal coefficients of the system in a system memory and one or more environmental variables in a physical space, wherein continuously monitoring comprises comparing the at least one variable to the at least one condition to determine whether the at least one condition has been met;
upon determination that one or more of the integrated conditions has been met, based on triggering of the one or more integrated conditions by data received from the at least one of the neural network, explainable artificial intelligence model, or machine learning algorithm via the behavioral modeling architecture separate from the explainable architecture, triggering a system-knowledge injection configured to maintain the behavior of the system within a predefined actual boundary;
identifying a gap between the predicted boundary of the system and the predefined actual boundary of the system;
in a case where the predicted boundary of the system extends outside of the predefined actual boundary, identifying the gap as an underspecified gap, and in a case where the predicted boundary of the system is inside of the predefined actual boundary, identifying the gap as a non-optimal gap;
performing one or more instances of tuning the behavioral modeling architecture, wherein tuning the behavioral modeling architecture comprises updating the predicted boundary based on information observed during the monitoring; and
finalizing the behavioral modeling architecture by terminating tuning of the behavioral modeling architecture upon determination that the gap between the predicted boundary of the system and the predefined actual boundary of the system is reduced.