Detecting source type mislabeling of machine data through determining prediction of source type through machine learning techniques
Implementations of the disclosure pertain to detecting mislabeling of a source type assigned to machine data through utilization of machine learning techniques. Operations of a computerized method for detecting the mislabeling include receiving machine data that has been assigned an initial source type upon receipt by a data intake and query system and parsing the data block into a plurality of events based on a source type definition of the initial source type. Further operations include generating a data representation of a first event being a portion of the machine data and is associated with a point in time, determining a predicted source type of the first event by at least analyzing the data representation through machine learning techniques, and performing a comparison between the predicted source type and the initial source type thereby determining whether the source type of the event was initially mislabeled.
1 . A computerized method comprising:
receiving a data block of machine data that is assigned an initial source type upon receipt;
parsing the data block into a plurality of events based on a source type definition of the initial source type and associating each of the plurality of events with the initial source type, wherein a first event includes a portion of the machine data and is associated with a point in time;
generating a data representation of the first event;
determining a predicted source type of the first event by at least analyzing the data representation through machine learning techniques; and
performing a comparison between the predicted source type and the initial source type thereby determining whether the source type of the event was initially mislabeled.
2 . The computerized method of claim 1 , wherein the machine learning techniques include use of a convoluted neural network (CNN) configured to analyze punctuation data of the first event against training data including previously obtained punctuation data.
3 . The computerized method of claim 1 , wherein the data block is received by a data intake and query system.
4 . The computerized method of claim 3 , wherein assigning the initial source type upon receipt of the data block is performed by an indexer or forwarder of the data intake and query system.
5 . The computerized method of claim 1 , wherein generating the data representation of the first event comprises extracting either a punctuation pattern or a character pattern from the first event, wherein the punctuation pattern includes a string of punctuation marks or the character pattern includes at least one or more special characters.
6 . The computerized method of claim 1 , further comprising:
responsive to determining that the source type of the first event was initially mislabeled, determining a source of the mislabeling.
7 . The computerized method of claim 6 , further comprising:
providing a recommendation for correcting instances of mislabeling based on the source of the mislabeling and a method used to determine the initial source type.
8 . A computing device, comprising:
one or more processors; and
a non-transitory computer-readable medium having stored thereon instructions that, when executed by the processor, cause the processor to perform operations including:
receiving a data block of machine data that is assigned an initial source type upon receipt;
parsing the data block into a plurality of events based on a source type definition of the initial source type and associating each of the plurality of events with the initial source type, wherein a first event includes a portion of the machine data and is associated with a point in time;
generating a data representation of the first event;
determining a predicted source type of the first event by at least analyzing the data representation through machine learning techniques; and
performing a comparison between the predicted source type and the initial source type thereby determining whether the source type of the event was initially mislabeled.
9 . The computing device of claim 8 , wherein the machine learning techniques include use of a convoluted neural network (CNN) configured to analyze punctuation data of the first event against training data including previously obtained punctuation data.
10 . The computing device of claim 8 , wherein the data block is received by a data intake and query system.
11 . The computing device of claim 10 , wherein assigning the initial source type upon receipt of the data block is performed by an indexer or forwarder of the data intake and query system.
12 . The computing device of claim 8 , wherein generating the data representation of the first event comprises extracting either a punctuation pattern or a character pattern from the first event, wherein the punctuation pattern includes a string of punctuation marks or the character pattern includes at least one or more special characters.
13 . The computing device of claim 8 , wherein the operations further including:
responsive to determining that the source type of the first event was initially mislabeled, determining a source of the mislabeling.
14 . The computing device of claim 13 , wherein the operations further including:
providing a recommendation for correcting instances of mislabeling based on the source of the mislabeling and a method used to determine the initial source type.
15 . The computing device of claim 8 , wherein the operations further including:
responsive to determining that the source type of the first event was initially mislabeled, determining a source of the mislabeling; and
providing a recommendation for correcting instances of mislabeling based on the source of the mislabeling and a method used to determine the initial source type.
16 . A non-transitory storage medium having stored thereon instructions that, when executed, cause performance of operations including:
receiving a data block of machine data that is assigned an initial source type upon receipt;
parsing the data block into a plurality of events based on a source type definition of the initial source type and associating each of the plurality of events with the initial source type, wherein a first event includes a portion of the machine data and is associated with a point in time;
generating a data representation of the first event;
determining a predicted source type of the first event by at least analyzing the data representation through machine learning techniques; and
performing a comparison between the predicted source type and the initial source type thereby determining whether the source type of the event was initially mislabeled.
17 . The non-transitory storage medium of claim 16 , wherein the machine learning techniques include use of a convoluted neural network (CNN) configured to analyze punctuation data of the first event against training data including previously obtained punctuation data.
18 . The non-transitory storage medium of claim 16 , wherein the data block is received by a data intake and query system.
19 . The non-transitory storage medium of claim 18 , wherein assigning the initial source type upon receipt of the data block is performed by an indexer or forwarder of the data intake and query system.
20 . The non-transitory storage medium of claim 16 , wherein generating the data representation of the first event comprises extracting either a punctuation pattern or a character pattern from the first event, wherein the punctuation pattern includes a string of punctuation marks or the character pattern includes at least one or more special characters.