Artificial intelligence-based automated parser creation
A method for artificial intelligence-based automated parser creation includes obtaining a first event log of one or more first event logs of first telemetry data. The first event log includes one or more event log key-value pairs. The method includes generating, using a first artificial intelligence (AI) model, a portion of parser code to map a first event log key of an event log key-value pair of the one or more event log key-value pairs to a predefined field. The method includes generating an event log parser that includes the portion of the parser code. The method includes causing the event log parser to be executed on a second event log of one or more second event logs of second telemetry log data.
1. A method, comprising:
obtaining a first event log of a first plurality of event logs of first telemetry data, wherein the first event log comprises a plurality of event log key-value pairs;
generating, using a first artificial intelligence (AI) model, a portion of parser code that converts the first event log to a predefined format by mapping a first event log key of a first event log key-value pair of the plurality of event log key-value pairs to a predefined field;
generating an event log parser that comprises the portion of the parser code; and
causing the event log parser to be executed on a second event log of a second plurality of event logs of second telemetry data.
2. The method of claim 1 , wherein:
the second event log comprises a second event log key and a corresponding value, wherein the second event log key corresponds to the first event log key; and
causing the event log parser to be executed on the second event log comprises the event log parser generating a data object that includes the predefined field and the value.
3. The method of claim 1 , wherein the first AI model comprises at least one of:
a large language model (LLM); or
a natural language processing (NLP)-based AI model.
4. The method of claim 1 , wherein the portion of parser code comprises source code configured to be compiled into computer-readable instructions.
5. The method of claim 1 , wherein the first AI model comprises an AI model trained on first training data comprising a first plurality of portions of parser code.
6. The method of claim 5 , wherein the first training data further comprises a plurality of sample event logs corresponding to the first plurality of portions of parser code.
7. The method of claim 6 , wherein the first training data further comprises a plurality of predefined event types for the plurality of sample event logs.
8. The method of claim 5 , wherein:
the first plurality of portions of parser code correspond to a first time; and
the method further includes replacing the first AI model with a second AI model, wherein the second AI model comprises an AI model trained on second training data comprising a second plurality of portions of parser code that correspond to a second time, and wherein the second time occurs after the first time.
9. The method of claim 1 , wherein the first AI model comprises an AI model trained on training data comprising a schema that includes a plurality of predefined fields.
10. A system comprising:
a memory; and
at least one processing device, coupled to the memory, configured to perform operations, comprising:
obtaining a first event log of a first plurality of event logs of first telemetry data, wherein the first event log comprises a plurality of event log key-value pairs;
identifying, from among a plurality of predefined fields and by using a clustering-based artificial intelligence (AI) model, a predefined field for an event log key of a first event log key-value pair of the plurality of event log key-value pairs;
generating a portion of parser code that converts the first event log to a predefined format by mapping the event log key to the identified predefined field;
generating an event log parser that comprises the portion of the parser code; and
causing the event log parser to be executed on a second event log of a second plurality of event logs of second telemetry data.
11. The system of claim 10 , wherein identifying the predefined field for the event log key comprises calculating a text string distance comparison metric between the event log key and the identified predefined field.
12. The system of claim 10 , wherein identifying the predefined field for the event log key comprises using a k-means clustering calculation.
13. The system of claim 10 , wherein identifying the predefined field for the event log key comprises using a fuzzy clustering calculation.
14. The system of claim 10 , wherein identifying the predefined field for the event log key comprises using a clustering calculation based on a corresponding value for event log key.
15. A non-transitory computer-readable storage medium comprising instructions for a computing device that, when executed by a processing device, cause the processing device to perform operations comprising:
obtaining a first event log of a first plurality of event logs of first telemetry data, wherein the first event log comprises a plurality of event log key-value pairs;
identifying, from among a plurality of predefined fields and by using an artificial intelligence (AI) model, a first predefined field for an event log key of a first event log key-value pair of the plurality of event log key-value pairs;
generating a portion of parser code that converts the first event log to a predefined format by mapping the event log key to the identified first predefined field;
generating an event log parser that comprises the portion of the parser code; and
causing the event log parser to be executed on a second event log of a second plurality of event logs of second telemetry data.
16. The computer-readable storage medium of claim 15 , wherein the AI model comprises an AI model trained on training data comprising a plurality of portions of parser code.
17. The computer-readable storage medium of claim 15 , wherein identifying the first predefined field for the event log key comprises calculating a text string distance comparison metric between the event log key and the identified first predefined field.
18. The computer-readable storage medium of claim 15 , wherein identifying the first predefined field for the event log key comprises:
for each predefined field in the plurality of predefined fields, calculating, using the AI model, a score for the predefined field;
presenting, on a user interface, the plurality of predefined fields, wherein a displayed order of the plurality of predefined fields is based on their respective calculated scores; and
obtaining an input, from the user interface, indicating the first predefined field.
19. The computer-readable storage medium of claim 18 , wherein obtaining the input indicating the first predefined field comprises obtaining text data input into the user interface.
20. The computer-readable storage medium of claim 18 , the operations further comprising updating the AI model based on the first predefined field.