Systems and methods for parsing log files using classification and plurality of neural networks

1. A system for parsing unstructured data comprising:

at least one processor; and

at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the system to perform operations comprising:

receiving unstructured data;

applying a classifier to the unstructured data to identify a type of the unstructured data;

extracting, by the classifier, one or more feature vectors from the unstructured data;

applying a neural network selector to the extracted one or more feature vectors;

comparing, by the neural network selector, the one or more extracted feature vectors with one or more representative feature vectors, wherein the one or more representative feature vectors are associated with a plurality of neural networks;

based on the comparison, selecting a corresponding neural network from the plurality of neural networks;

training the selected corresponding neural network using a character window;

applying the selected neural network to the unstructured data to obtain structured data; and

outputting the structured data.

2. The system of claim 1 , wherein the structured data comprises at least one of relational data, graphical data, or object-oriented data.

3. The system of claim 1 , wherein the unstructured data comprises a log file.

4. The system of claim 3 , wherein the log file is generated by at least one application.

5. The system of claim 1 , wherein the classifier comprises at least one of a linear classifier, a bag-of-words model, or a character-level convolutional neural network.

6. The system of claim 1 , wherein the classifier identifies the type by searching the unstructured data for one or more known keys associated with the type.

7. The system of claim 1 , wherein the one or more known keys comprise at least one alphanumeric string associated with the type.

8. The system of claim 1 , wherein the corresponding neural network is indexed by the type and retrieved using the index.

9. The system of claim 1 , wherein the corresponding neural network comprises at least one of a recurrent neural network or a convolutional neural network.

10. The system of claim 1 , wherein the character window comprises a range between characters.

11. The system of claim 1 , wherein the corresponding neural network comprises one of a plurality of neural networks, and each neural network has an associated character window.

12. A system for parsing unstructured data comprising:

at least one processor; and

at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the system to perform operations comprising:

receiving a log file comprising unstructured data;

applying a classifier to the unstructured data to identify a distribution of one or more letters, a distribution of one or more integers, a distribution of one or more special characters, or a distribution of one or more alphanumeric characters;

applying a neural network selector to the identified distribution;

comparing, by the neural network selector, the identified distribution with one or more representative distributions, wherein the one or more representative distributions are associated with a plurality of neural networks;

based on the comparison, selecting from the plurality of neural networks a corresponding neural network;

training the selected corresponding neural network using a character window;

applying the selected neural network to the log file to obtain structured data corresponding to the unstructured data of the log file; and

outputting the structured data.

13. The system of claim 12 , wherein the structured data comprises at least one of relational data, graphical data, or object-oriented data.

14. The system of claim 12 , wherein outputting the structured data comprises at least one of storing the structured data or transmitting the structured data to an external device.

15. The system of claim 12 , wherein selecting the corresponding neural network comprises determining a neural network having an associated distribution range, wherein the distribution identified by the classifier is within the range.

16. The system of claim 12 , wherein selecting the corresponding neural network comprises determining a neural network having an associated distribution that is within a threshold of the distribution identified by the classifier.

17. The system of claim 12 , wherein the classifier identifies a plurality of distributions, and wherein selecting the corresponding neural network comprises determining a neural network having associated distribution ranges encompassing the distributions identified by the classifier.

18. The system of claim 12 , wherein selecting the corresponding neural network comprises determining a neural network having associated distributions that are within corresponding thresholds of the distributions identified by the classifier.

19. The system of claim 12 , wherein selecting the corresponding neural network comprises determining a neural network having an associated loss on the received log file within a corresponding predefined threshold.

20. A system for parsing unstructured data comprising:

at least one processor; and

at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the system to perform operations comprising:

receiving a log file comprising unstructured data;

pre-processing the log file to standardize delimiters within the unstructured data;

applying a classifier to the unstructured data to identify an application name having generated the log file;

applying a neural network selector to the identified application name;

comparing, by the neural network selector, the identified application name with one or more representative application names, wherein the one or more application names is associated with a plurality of neural networks;

based on the comparison, selecting a corresponding neural network trained to parse log files from the application;

training the selected corresponding neural network using a character window;

applying the selected neural network to the log file to obtain tabular data corresponding to the unstructured data of the log file; and

outputting the tabular data to a relational database.