Event views in data intake stage of machine data processing platform
View Patent ↗A security platform employs a variety techniques and mechanisms to detect security related anomalies and threats in a computer network environment. The security platform is “big data” driven and employs machine learning to perform security analytics. The security platform performs user/entity behavioral analytics (UEBA) to detect the security related anomalies and threats, regardless of whether such anomalies/threats were previously known. The security platform can include both real-time and batch paths/modes for detecting anomalies and threats. By visually presenting analytical results scored with risk ratings and supporting evidence, the security platform enables network security administrators to respond to a detected anomaly or threat, and to take action promptly.
1. A method of enabling, on a computer platform, homogeneous access to event data, the method comprising:
receiving, by a computer platform, event data representing a plurality of events on a computer network, each of the events corresponding to at least one category of a plurality of event categories; and
automatically editing the event data of each of the plurality of events to associate each of the events with at least one of a plurality of distinct, predefined, non-graphical programming access interfaces, based on an event category to which the event belongs, wherein said automatically editing the event data includes adding, to the event data of each event, a view identifier that corresponds uniquely to a particular one of the plurality of programming access interfaces, such that each of the plurality of programming access interfaces is subscribable by each of a plurality of programmatic entities by designation of the corresponding view identifier, wherein each of the plurality of programming access interfaces is defined to include a number of fields and/or a number of methods, said fields and/or methods allowing a downstream programmatic entity, by designation of the view identifier of any of the programming access interfaces together with a field or a method of said programming access interface, to receive information about an event, wherein the information corresponds to the designated view identifier and to the designated field or method,
wherein each of the plurality of programming access interfaces is configured to extract, for a designated field, a particular set of information from the event data and/or to generate, for a designated method, a particular set of information based on the event data.
2. The method of claim 1 , wherein at least one of the plurality of programming access interfaces further includes an input for controlling the action that the logic performs.
3. The method of claim 1 , wherein the information only includes a defined subset of a complete set of the event data representing an event.
4. The method of claim 1 ,
wherein event data representing the plurality of events are in different data formats, and wherein said method provides a uniform way to access information about the plurality of events.
5. The method of claim 1 , further comprising:
wherein said adding is performed for event data representing each of a plurality of events that belong to a same category to enable, via the interface, a uniform way to access information about the plurality of events.
6. The method of claim 1 , further comprising:
determining an event category to which the event belongs, from the plurality of event categories, based on the event data, wherein each said view identifier and programming access interface corresponds to the event category.
7. The method of claim 1 , further comprising:
determining an event category to which the event belongs, from the plurality of event categories, based on a type of a machine that generated the event, wherein each said view identifier and programming access interface corresponds to the event category.
8. The method of claim 1 , wherein each said view identifier and programming access interface corresponds to an event category.
9. The method of claim 1 , further comprising:
determining an event category, from the plurality of event categories, based on a type of a machine that generated the event, wherein each said view identifier and programming access interface corresponds to the event category, the event category characterizing events generated by a plurality of types of machines.
10. The method of claim 1 , further comprising: parsing the event data based on a predetermined data format.
11. The method of claim 1 , further comprising: parsing the event data based on a predetermined data format that specifies which data in an event represent a key or a value.
12. The method of claim 1 , wherein each said view identifier is added as a field in an event.
13. The method of claim 1 , further comprising:
identifying a plurality of attributes of an event, based on the event data,
wherein the information includes a defined subset of the plurality of attributes of an event.
14. The method of claim 1 , further comprising:
identifying a plurality of attributes of an event, based on the event data,
wherein the information includes a defined subset of the plurality of attributes of an event,
wherein the attributes include at least one of: a key, a value, or a key-value pair.
15. The method of claim 1 , further comprising:
performing, by the downstream programmatic entity, analytics on event data representing the plurality of events using information about the plurality of events.
16. The method of claim 1 , further comprising:
performing, by the downstream programmatic entity, analytics on event data representing the plurality of events using information about the plurality of events,
wherein information accessible via the programming access interface includes at least one of: (1) information generated by logic included in the programming access interface, the logic being operable to perform an action on the event data representing the plurality of events, or (2) a predefined subset of a complete set of the event data representing the plurality of events.
17. The method of claim 1 , further comprising:
using at least one said view identifier to determine whether an event is relevant to the downstream programmatic entity.
18. The method of claim 1 , further comprising:
automatically routing the information to the downstream programmatic entity having designated a corresponding view identifier.
19. The method of claim 1 , further comprising:
automatically routing the information to the downstream programmatic entity having designated a corresponding view identifier,
wherein the downstream programmatic entity is configured to identify security-oriented anomalies represented by an event.
20. The method of claim 1 , wherein the event data comprises machine data.
21. The method of claim 1 , wherein the event data comprises timestamped machine data.
22. The method of claim 1 , further comprising:
configuring said adding step by making an adjustment to a configuration file.
23. The method of claim 1 , wherein the method is performed as part of an extract-transform-load stage of at least one of a distributed event processing system or an anomaly detection system.
24. A computer system comprising:
a communication device; and
a processor configured to cause the computer system to:
receive, from the communication device, event data representing a plurality of events on a computer network, each of the events corresponding to at least one category of a plurality of event categories; and
automatically edit the event data of each of the plurality of events to associate each of the events with at least one of a plurality of distinct, predefined, non-graphical programming access interfaces, based on an event category to which the event belongs, wherein automatically editing the event data includes adding, to the event data of each event, a view identifier that corresponds uniquely to a particular one of the plurality of programming access interfaces, such that each of the plurality of programming access interfaces is subscribable by each of a plurality of programmatic entities by designation of the corresponding view identifier, wherein each of the plurality of programming access interfaces is defined to include a number of fields and/or a number of methods, said fields and/or methods allowing a downstream programmatic entity, by designation of the view identifier of any of the programming access interfaces together with a field or a method of said programming access interface, to receive information about an event, wherein the information corresponds to the designated view identifier and to the designated field or method,
wherein each of the plurality of programming access interfaces is configured to extract, for a designated field, a particular set of information from the event data and/or to generate, for a designated method, a particular set of information based on the event data.
25. The computer system of claim 24 , wherein at least one of said programming access interfaces further includes logic operable to perform an action on the event data representing the event to generate the information.
26. The computer system of claim 24 , wherein the information only includes a defined subset of a complete set of the event data representing an event.
27. A non-transitory machine-readable storage medium for use in a processing system, the non-transitory machine-readable storage medium storing instructions, execution of which in the processing system causes the processing system to perform operations comprising:
receiving event data representing a plurality of events on a computer network, each of the events corresponding to at least one category of a plurality of event categories; and
automatically editing the event data of each of the plurality of events to associate each of the events with at least one of a plurality of distinct, predefined, non-graphical programming access interfaces, based on an event category to which the event belongs, wherein said automatically editing the event data includes adding, to the event data of each event, a view identifier that corresponds uniquely to a particular one of the plurality of programming access interfaces, such that each of the plurality of programming access interfaces is subscribable by each of a plurality of programmatic entities by designation of the corresponding view identifier, wherein each of the plurality of programming access interfaces is defined to include a number of fields and/or a number of methods, said fields and/or methods allowing a downstream programmatic entity, by designation of the view identifier of any of the programming access interfaces together with a field or a method of said programming access interface, to receive information about an event, wherein the information corresponds to the designated view identifier and to the designated field or method,
wherein each of the plurality of programming access interfaces is configured to extract, for a designated field, a particular set of information from the event data and/or to generate, for a designated method, a particular set of information based on the event data.