System and method for anomaly detection interpretation
A system and method for anomaly interpretation and mitigation. A method includes extracting at least one input feature vector from observation data related to an observation; applying an isolation forest to the at least one input feature vector, wherein the isolation forest includes a plurality of estimators, wherein each estimator is a decision tree, wherein the output of each estimator is a split-path of a plurality of split-paths, each split-path having a path-length and including name and a corresponding value for a respective output feature of a plurality of output features; generating a mapping object based on the application of the isolation forest to the at least one feature vector, wherein the mapping object includes the plurality of split-paths; clipping the mapping object based on the path-length of each split-path; and determining at least one mitigation action based on the clipped mapping object.
1. A method for anomaly interpretation and mitigation, comprising:
extracting an input feature vector from observation data indicating anomalous behavior of a connected device;
applying an isolation forest to the input feature vector, wherein the isolation forest includes a plurality of estimators, wherein each estimator is a decision tree, wherein an output of each estimator is a split-path of a plurality of split-paths, each split-path having a path-length and including name and a corresponding value for a respective output feature of a plurality of output features, wherein each output feature represents at least a portion of a description of why the observation was determined to indicate anomalous behavior;
generating a mapping object based on an application of the isolation forest to the feature vector, wherein the mapping object includes the plurality of split-paths;
removing a number of split-paths from the mapping object based at least in part on a predetermined ratio of a total number of the plurality of estimators;
determining additional context data determined from remaining split-paths; and
determining at least one mitigation action.
2. The method of claim 1 , further comprising:
sorting the plurality of split-paths based on their respective path-lengths; and removing at least one split-path from the plurality of split-paths.
3. The method of claim 2 , further comprising:
determining the at least one split-path to be removed from the plurality of split-paths based on the sorted plurality of split-paths and a ratio of a total number of estimators.
4. The method of claim 1 , further comprising:
generating additional contextual data, wherein the at least one mitigation action is determined based further on the additional contextual data.
5. The method of claim 4 , wherein the additional contextual data includes statistical data for each output feature determined based on the remaining split-paths.
6. The method of claim 5 , wherein the statistical data indicates, for each output feature, at least one of: a number of occurrences of the output feature among the output features of the remaining split-paths, a percentile of the value of the output feature with respect to other observations in a sub-population of observations, and a position of the output feature within a distribution of the output features of the remaining split-paths.
7. The method of claim 1 , wherein the observation was determined to indicate an anomaly.
8. The method of claim 7 , wherein the observation has an anomaly score representing a likelihood that the observation indicates an anomaly, wherein the anomaly score is above a threshold.
9. The method of claim 1 , wherein each estimator is a binary decision tree.
10. A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to execute a process, the process comprising:
extracting at least one input feature vector from observation data related to an observation, wherein the observation indicates anomalous behavior of a connected device;
applying an isolation forest to the at least one input feature vector, wherein the isolation forest includes a plurality of estimators, wherein each estimator is a decision tree, wherein an output of each estimator is a split-path of a plurality of split-paths, each split-path having a path-length and including name and a corresponding value for a respective output feature of a plurality of output features, wherein each output feature represents at least a portion of a description of why the observation was determined to indicate anomalous behavior;
generating a mapping object based on an application of the isolation forest to the at least one feature vector, wherein the mapping object includes the plurality of split-paths;
removing a number of split-paths from the mapping objects based at least in part on a predetermined ratio of a total number of the plurality of estimators;
determining additional context data determined from remaining split-paths; and
determining at least one mitigation action.
11. A system for anomaly interpretation, comprising: a processing circuitry; and
a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:
extract at least one input feature vector from observation data related to an observation, wherein the observation indicates anomalous behavior of a connected device;
apply an isolation forest to the at least one input feature vector, wherein the isolation forest includes a plurality of estimators, wherein each estimator is a decision tree, wherein an output of each estimator is a split-path of a plurality of split-paths, each split-path having a path-length and including name and a corresponding value for a respective output feature of a plurality of output features, wherein each output feature represents at least a portion of a description of why the observation was determined to indicate anomalous behavior;
generate a mapping object based on an application of the isolation forest to the at least one feature vector, wherein the mapping object includes the plurality of split-paths;
remove a number of split-paths from the mapping object based at least in part on a predetermined ratio of a total number of the plurality of estimators;
determining additional context data determined from remaining split-paths; and
determine at least one mitigation action.
12. The system of claim 11 , wherein the system is further configured to:
sort the plurality of split-paths based on their respective path-lengths; and remove at least one split-path from the plurality of split-paths.
13. The system of claim 12 , wherein the system is further configured to:
determine the at least one split-path to be removed from the plurality of split-paths based on the sorted plurality of split-paths and a ratio of a total number of estimators.
14. The system of claim 11 , wherein the system is further configured to:
generate additional contextual data, wherein the at least one mitigation action is determined based further on the additional contextual data.
15. The system of claim 14 , wherein the additional contextual data includes statistical data for each output feature determined based on the remaining split-paths.
16. The system of claim 15 , wherein the statistical data indicates, for each output feature, at least one of: a number of occurrences of the output feature among the output features of the remaining split-paths, a percentile of the value of the output feature with respect to other observations in a sub-population of observations, and a position of the output feature within a distribution of the output features of the remaining split-paths.
17. The system of claim 11 , wherein the observation was determined to indicate an anomaly.
18. The system of claim 17 , wherein the observation has an anomaly score representing a likelihood that the observation indicates an anomaly, wherein the anomaly score is above a threshold.
19. The system of claim 11 , wherein each estimator is a binary decision tree.