Swappable online artificial intelligence algorithms implemented in a data intake and query system
Systems and methods are described for processing ingested data, detecting anomalies in the ingested data, and providing explanations of a possible cause of the detected anomalies as the data is being ingested. For example, a token or field in the ingested data may have an anomalous value. Tokens or fields from another portion of the ingested data can be extracted and analyzed to determine whether there is any correlation between the values of the extracted tokens or fields and the anomalous token or field having an anomalous value. If a correlation is detected, this information can be surfaced to a user.
1 . A method, comprising:
obtaining first raw machine data from an event data stream generated by one or more components in an information technology environment;
comparing, using a set of data, an accuracy of a first model and an accuracy of a second model by at least (i) comparing an output produced by the first model processing the set of data to an actual or expected output to generate a first data result being a first prediction of one or more values of the set of data, (ii) comparing an output produced by the second model processing the set of data to the actual or expected output to generate a second data result being a second prediction of the one or more values of the set of data, and (iii) comparing the first data result and the second data result to determine that the second model is more accurate with respect to the actual or expected output than the first model, wherein the first model is generated using a first artificial intelligence algorithm code, the first raw machine data, and a base model, and wherein the second model is generated using a second artificial intelligence algorithm code, the first raw machine data, and the base model;
responsive to determining that the second model is more accurate than the first model, obtaining second raw machine data from the event data stream generated by the one or more components in the information technology environment; and
processing the second raw machine data from the event data stream using the second model by swapping a portion of the first artificial intelligence algorithm code with a portion of the second artificial intelligence algorithm code,
wherein the second artificial intelligence algorithm code includes at least a second transformation operation and a second reference to a storage location of a model state of the second model and the first artificial intelligence algorithm code includes at least a first transformation operation and a reference to a storage location of a model state of the first model in lieu of variables defining the model state of the first model to avoid loss of the model state of the first model during swapping of the first artificial intelligence algorithm code for the second artificial intelligence algorithm code and avoid re-training of the second model using previously ingested raw machine data by the first artificial intelligence algorithm code, and
wherein the swapping of the portion of the first artificial intelligence algorithm code comprises swapping the first transformation operation with the second transformation operation in response to the determination that the second model is more accurate than the first model and the second artificial intelligence code utilizing the model state of the first model.
2 . The method of claim 1 , wherein the swapping of the portion of the first artificial intelligence algorithm code with the portion of the second artificial intelligence algorithm code further comprises swapping the first transformation operation with the second transformation operation in response to the determination that the second model is more accurate than the first model.
3 . The method of claim 1 , wherein the first model and the second model obtain the set of data from a source specified by a graph representing a data processing pipeline.
4 . The method of claim 1 , wherein the first model and the second model obtain the set of data from a source specified by a graph representing a data processing pipeline, and wherein a version of an output of the first model is written to an external storage system specified by the graph.
5 . The method of claim 1 , wherein the first model and the second model obtain the set of data from a source specified by a graph representing a data processing pipeline, wherein a version of an output of the first model is written to an external storage system specified by the graph, and wherein an output of the second model is not written to any external storage system until the second model is determined to be more accurate than the first model.
6 . The method of claim 1 , wherein the first model and the second model obtain the set of data from a source specified by a graph representing a data processing pipeline, wherein a version of an output of the first model is written to an external storage system specified by the graph, wherein an output of the second model is not written to any external storage system until the second model is determined to be more accurate than the first model, wherein comparing an accuracy of the first model and an accuracy of the second model further comprises:
determining, a time period after the second model is generated, whether to continue writing the version of the output of the first model to the external storage system or whether to begin writing a version of the output of the second model to the external storage system; and
comparing the accuracy of the first model and the accuracy of the second model to determine which version of output to write to the external storage system.
7 . The method of claim 1 , wherein comparing an accuracy of the first model and an accuracy of the second model further comprises:
obtaining a set of subsequent raw machine data being the set of data from the event data stream;
generating the output produced by the first model corresponding to one or more first predictions associated with the set of subsequent raw machine data in response to an application of the set of subsequent raw machine data as an input to the first model;
generating the output produced by the second model corresponding to one or more second predictions associated with the set of subsequent raw machine data in response to an application of the set of subsequent raw machine data as an input to the second model; and
comparing an accuracy of the one or more first predictions to the actual or expected output to an accuracy of the one or more second predictions to the actual or expected output.
8 . The method of claim 1 , wherein comparing an accuracy of the first model and an accuracy of the second model further comprises:
obtaining a set of subsequent raw machine data being the set of data from the event data stream that represents raw machine data obtained from the event stream over a threshold period of time;
generating the output produced by the first model corresponding to one or more first predictions associated with the set of subsequent raw machine data in response to an application of the set of subsequent raw machine data as an input to the first model;
generating the output produced by the second model corresponding to one or more second predictions associated with the set of subsequent raw machine data in response to an application of the set of subsequent raw machine data as an input to the second model; and
comparing an accuracy of the one or more first predictions to the actual or expected output to an accuracy of the one or more second predictions to the actual or expected output.
9 . The method of claim 1 , wherein comparing an accuracy of the first model and an accuracy of the second model further comprises comparing a loss associated with the first model corresponding to the first data result and a loss associated with the second model corresponding to the second data result.
10 . The method of claim 1 , further comprising:
obtaining a set of subsequent raw machine data being the set of data from the event data stream;
generating, in a production stack, the output produced by the first model corresponding to one or more first predictions associated with the set of subsequent raw machine data in response to an application of the set of subsequent raw machine data as an input to the first model;
generating, in a test stack separate from the production stack, the output produced by the second model corresponding to one or more second predictions associated with the set of subsequent raw machine data in response to an application of the set of subsequent raw machine data as an input to the second model; and
generating, in the production stack, a third prediction the third raw machine data and the second model.
11 . The method of claim 1 , wherein processing the second raw machine data from the event data stream using the second model further comprises:
swapping the first model with the second model in a production stack; and
processing the second raw machine data and subsequent raw machine data using the second model in the production stack.
12 . The method of claim 1 , wherein a data ingestion pipeline comprises an operator that implements the first artificial intelligence algorithm code, and wherein the method further comprises refreshing the data ingestion pipeline to replace the operator with a second operator that implements the second artificial intelligence algorithm code.
13 . The method of claim 1 , wherein a data ingestion pipeline comprises an operator that implements the first artificial intelligence algorithm code, and wherein the method further comprises:
refreshing the data ingestion pipeline to replace the operator with a second operator that implements the second artificial intelligence algorithm code; and
processing the second raw machine data and subsequent raw machine data in the data ingestion pipeline using the second operator.
14 . A system, comprising:
one or more data stores including computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions, wherein execution of the computer-executable instructions causes the system to:
obtain first raw machine data from an event data stream generated by one or more components in an information technology environment;
compare, using a set of data, an accuracy of a first model and an accuracy of a second model by at least (i) comparing an output produced by the first model processing the set of data to an actual or expected output to generate a first data result being a first prediction of one or more values of the set of data, (ii) comparing an output produced by the second model processing the set of data to the actual or expected output to generate a second data result being a second prediction of the one or more values of the set of data, and (iii) comparing the first data result and the second data result to determine that the second model is more accurate with respect to the actual or expected output than the first model, wherein the first model is generated using a first artificial intelligence algorithm code, the first raw machine data, and a base model, and wherein the second model is generated using a second artificial intelligence algorithm code, the first raw machine data, and the base model;
responsive to determination that the second model is more accurate than the first model, obtain second raw machine data from the event data stream generated by the one or more components in the information technology environment; and
process the second raw machine data from the event data stream using the second model by swapping a portion of the first artificial intelligence algorithm code with a portion of the second artificial intelligence algorithm code,
wherein the second artificial intelligence algorithm code includes at least a second transformation operation and a second reference to a storage location of a model state of the second model and the first artificial intelligence algorithm code includes at least a first transformation operation and a reference to a storage location of a model state of the first model in lieu of variables defining the model state of the first model to avoid loss of the model state of the first model during swapping of the first artificial intelligence algorithm code for the second artificial intelligence algorithm code and avoid re-training of the second model using previously ingested raw machine data by the first artificial intelligence algorithm code, and
wherein the swapping of the portion of the first artificial intelligence algorithm code comprises swapping the first transformation operation with the second transformation operation in response to the determination that the second model is more accurate than the first model and the second artificial intelligence code utilizing the model state of the first model.
15 . The system of claim 14 , wherein the first artificial intelligence algorithm code further comprises a transformation operation and a reference to a storage location of a model state of the first model.
16 . The system of claim 14 , wherein a data ingestion pipeline comprises an operator that implements the first artificial intelligence algorithm code, and wherein execution of the computer-executable instructions further causes the system to:
refresh the data ingestion pipeline to replace the operator with a second operator that implements the second artificial intelligence algorithm code; and
process the second raw machine data and subsequent raw machine data in the data ingestion pipeline using the second operator.
17 . A non-transitory computer-readable media comprising instructions executable by one or more processors of a computing system to:
obtain first raw machine data from an event data stream generated by one or more components in an information technology environment;
compare, using a set of data, an accuracy of a first model and an accuracy of a second model by at least (i) comparing an output produced by the first model processing the set of data to an actual or expected output to generate a first data result being a first prediction of one or more values of the set of data, (ii) comparing an output produced by the second model processing the set of data to the actual or expected output to generate a second data result being a second prediction of the one or more values of the set of data, and (iii) comparing the first data result and the second data result to determine that the second model is more accurate with respect to the actual or expected output than the first model, wherein the first model is generated using a first artificial intelligence algorithm code, the first raw machine data, and a base model, and wherein the second model is generated using a second artificial intelligence algorithm code, the first raw machine data, and the base model;
determine that the second model is more accurate than the first model;
obtain second raw machine data from the event data stream generated by the one or more components in the information technology environment; and
process the second raw machine data from the event data stream using the second model by swapping a portion of the first artificial intelligence algorithm code with a portion of the second artificial intelligence algorithm code,
wherein the second artificial intelligence algorithm code includes at least a second transformation operation and a second reference to a storage location of a model state of the second model and the first artificial intelligence algorithm code includes at least a first transformation operation and a reference to a storage location of a model state of the first model in lieu of variables defining the model state of the first model to avoid loss of the model state of the first model during swapping of the first artificial intelligence algorithm code for the second artificial intelligence algorithm code and avoid re-training of the second model using previously ingested raw machine data by the first artificial intelligence algorithm code, and
wherein the swapping of the portion of the first artificial intelligence algorithm code comprises swapping the first transformation operation with the second transformation operation in response to the determination that the second model is more accurate than the first model and the second artificial intelligence code utilizing the model state of the first model.
18 . The non-transitory computer-readable media of claim 17 , wherein the swapping of the portion of the first artificial intelligence algorithm code with the portion of the second artificial intelligence algorithm code further comprises swapping the first transformation operation with the second transformation operation in response to the determination that the second model is more accurate than the first model.
19 . The non-transitory computer-readable media of claim 17 , wherein the first model and the second model obtain the set of data from a source specified by a graph representing a data processing pipeline.
20 . The non-transitory computer-readable media of claim 17 , wherein the first model and the second model obtain the set of data from a source specified by a graph representing a data processing pipeline, and wherein a version of an output of the first model is written to an external storage system specified by the graph.