Machine learning-based data analyses for outlier detection
Techniques are described for providing a machine learning (ML) data analytics application including guided ML workflows that facilitate the end-to-end training and use of various types of ML models, where such guided workflows may also be referred to as ML “experiments.” One such model is an outlier detection model to assist in the monitoring of computer network traffic and computer performance. For example, the ML data analytics application may generate an outlier detection model using user-identified data from a data source and parameter information. The generates outlier detection model can include distribution functions of distribution types selected from a plurality of distribution types by a distribution fitting algorithm.
1. A computer-implemented method comprising:
causing display of a first user interface component, the first user interface component enabling user identification of data to be used to generate an outlier detection model using an outlier detection algorithm;
obtaining the data from a data source;
causing display of a second user interface component, the second user interface component enabling user indication of parameter information related to the outlier detection model;
generating the outlier detection model based on the data from the data source and the parameter information related to the outlier detection model; and
causing display of a third user interface component, the third user interface component including information about the outlier detection model.
2. The computer-implemented method of claim 1 , wherein the outlier detection model includes a distribution function of a distribution type selected from a plurality of distribution types by a distribution fitting algorithm.
3. The computer-implemented method of claim 2 , further comprising:
identifying, using the distribution fitting algorithm, the distribution function from a plurality of distribution functions, the distribution function having a highest likelihood of matching the data from the data source; and
selecting the distribution function for display on the second user interface component.
4. The computer-implemented method of claim 2 , wherein the plurality of distribution types include a Gaussian distribution, a Gaussian kernel density estimation (KDE) distribution, and an exponential distribution.
5. The computer-implemented method of claim 1 , further comprising:
receiving input, via the second user interface component, identifying a field contained in the data, wherein the outlier detection model is generated based on values associated with the field, and wherein the second user interface component includes a visualization showing one or more distribution functions modeled based on the values associated with the field.
6. The computer-implemented method of claim 5 , wherein the visualization showing the one or more distribution functions includes icons indicating locations of outlier data points for each of the one or more distribution functions.
7. The computer-implemented method of claim 5 , wherein the second user interface component enables a user to select a number of the one or more distribution functions to display in the visualization.
8. The computer-implemented method of claim 1 , wherein the first user interface component further enables a user to identify one or more of: a user-specified search query, one or more predefined datasets, or one or more predefined metrics.
9. The computer-implemented method of claim 1 , wherein the data source is a data store of a data intake and query system storing timestamped event data.
10. The computer-implemented method of claim 1 , wherein the parameter information related to the outlier detection model includes a field to analyze, a set of selectable options to define data subsets of the data from the data source associated with the field to analyze, a distribution type, and an outlier tolerance threshold.
11. The computer-implemented method of claim 10 , wherein the outlier tolerance threshold is an upper bound on a percentage of data points from the data being numerical outliers.
12. The computer-implemented method of claim 10 , further comprising:
receiving input, via the second user interface component, modifying the outlier tolerance threshold to produce a modified outlier tolerance threshold; and
in response to receiving the input, modifying the outlier detection model based on the modified outlier tolerance threshold.
13. The computer-implemented method of claim 10 , wherein the parameter information further includes a time feature, and wherein the method further comprises:
receiving, via the second user interface component, a selection to enable the time feature; and
in response to receiving the selection to enable the time feature, modifying the set of selectable options to define data subsets of the data from the data source associated with the field to analyze to include time options.
14. The computer-implemented method of claim 13 , wherein the outlier detection model further includes a time series representation of the data from the data source based on the parameter information related to the outlier detection model, the parameter information including a selected time option from the time options.
15. The computer-implemented method of claim 1 , wherein data representing the outlier detection model is stored in a data store of a data intake and query system.
16. The computer-implemented method of claim 1 , further comprising:
causing display of a fourth user interface component, the fourth user interface component enabling user selection of a first data subset for comparison with a second data subset; and
generating comparison data of the first data subset and the second data subset.
17. The computer-implemented method of claim 16 , wherein the comparison data includes a score indicating a relationship between the first data subset and the second data subset.
18. The computer-implemented method of claim 16 , wherein the first data subset is a first distribution function in the outlier detection model and the second data subset is a second distribution function in the outlier detection model.
19. The computer-implemented method of claim 18 , wherein the fourth user interface component includes a visualization of an equality of the first distribution function and the second distribution function.
20. The computer-implemented method of claim 16 , wherein the first data subset comprises a first distribution function in a first outlier detection model and the second data subset comprises a second distribution function in a second outlier detection model.
21. The computer-implemented method of claim 16 , wherein the first data subset is a first distribution function and the second data subset is a field.
22. The computer-implemented method of claim 1 , further comprising causing display of a fourth user interface component, the fourth user interface component including user interface elements enabling one or more of: deploying the outlier detection model to a data intake and query system, creating an alert associated with the outlier detection model, and scheduling additional training of the outlier detection model.
23. The computer-implemented method of claim 1 , wherein the first user interface component, the second user interface component, and the third user interface component are part of a guided workflow comprising a set of graphical user interfaces (GUIs) including user interface elements enabling navigation among the set of GUIs.
24. The computer-implemented method of claim 1 , further comprising:
causing concurrent display of a first machine learning (ML) workflow progress indicator and the first user interface component, concurrent display of a second ML workflow progress indicator and the second user interface component, and concurrent display of a third ML workflow progress indicator and the third user interface component.
25. The computer-implemented method of claim 2 , wherein the third user interface component includes a visualization showing a histogram generated based on data used to generate the distribution function.
26. A non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors, cause performance of operations comprising:
causing display of a first user interface component, the first user interface component enabling user identification of data to be used to generate an outlier detection model using an outlier detection algorithm;
obtaining the data from a data source;
causing display of a second user interface component, the second user interface component enabling user indication of parameter information related to the outlier detection model;
generating the outlier detection model based on the data from the data source and the parameter information related to the outlier detection model; and
causing display of a third user interface component, the third user interface component including information about the outlier detection model.
27. The non-transitory computer-readable storage medium of claim 26 , wherein the outlier detection model includes a distribution function of a distribution type selected from a plurality of distribution types by a distribution fitting algorithm.
28. The non-transitory computer-readable storage medium of claim 27 , wherein the instructions further cause performance of operations comprising:
identifying, using the distribution fitting algorithm, the distribution function from a plurality of distribution functions, the distribution function having a highest likelihood of matching the data from the data source; and
selecting the distribution function for display on the second user interface component.
29. The non-transitory computer-readable storage medium of claim 26 , wherein the instructions further cause performance of operations comprising:
receiving input, via the second user interface component, identifying a field contained in the data, wherein the outlier detection model is generated based on values associated with the field, and wherein the second user interface component includes a visualization showing one or more distribution functions modeled based on the values associated with the field.
30. An apparatus, comprising:
one or more processors;
a non-transitory computer-readable storage medium storing instructions which, when executed by the one or more processors, causes the apparatus to:
cause display of a first user interface component, the first user interface component enabling user identification of data to be used to generate an outlier detection model using an outlier detection algorithm;
obtain the data from a data source;
causing display of a second user interface component, the second user interface component enabling user indication of parameter information related to the outlier detection model;
generate the outlier detection model based on the data from the data source and the parameter information related to the outlier detection model; and
cause display of a third user interface component, the third user interface component including information about the outlier detection model.