Method for data reduction in a computer network security system
A method including collecting and aligning raw data from a plurality of network nodes, wherein dissimilar data types are aligned as input events; filtering the input events by discarding events and/or parts of events that are detected to be equal or similar to previously observed events or events and/or parts of events found to be redundant by using predetermined criteria; separating processing of the input events into event aggregation and event enrichment processes, wherein the event aggregation process includes processing all the input events for generating aggregated events, and the event enrichment process includes processing only events passed by the filtering and the aggregated events from the event aggregation process; and analysing the data received from the event enrichment process for generating a security related decision.
1. A method in a computer network security system, the method comprising:
collecting and aligning raw data from a plurality of network nodes, wherein dissimilar data types are aligned as input events;
filtering the input events by discarding events and/or parts of events that are detected to be equal or similar to previously observed events or events and/or parts of events found to be redundant by using predetermined criteria;
separating processing of the input events into event aggregation and event enrichment processes, wherein
the event aggregation process comprises processing all the input events for generating aggregated events, and
the event enrichment process comprises processing only events passed by the filtering and the aggregated events from the event aggregation process; and
analysing the data received from the event enrichment process for generating a security related decision.
2. The method according to claim 1 , wherein filtering of the input events is based on a lookup logic defining processes for at least one of: generation of a lookup cache collecting information about already observed events, performing lookup queries to the lookup cache for checking which events have been observed already, determining essential elements of the passed through events required for the analysis, determining unnecessary elements of the passed through events to be culled.
3. The method according to claim 2 , the method further comprising dynamically controlling the lookup logic on the basis of the analysis.
4. The method according to claim 1 , wherein the filtering process further comprises determining for each input event a level of uniqueness on the basis of predetermined event elements, and using the determined level of uniqueness to decide whether an input event has been observed before.
5. The method according to claim 1 , wherein said event aggregation process comprises aggregating information about the input events to support the creation of different views according to specific types of entities related to the input events.
6. The method according to claim 1 , wherein said event aggregation process comprises creating aggregated events on the basis of predetermined event elements of the input events.
7. The method according to claim 1 , wherein said event enrichment process comprises extending the structure and context of events with previously collected data.
8. The method according to claim 1 , wherein said analysing the data comprises using at least one of the following processes for generating the decision:
predetermined rules, heuristics, machine learning models, fuzzy logic based models, statistical inference based model.
9. The method according to claim 1 , in case the generated security related decision establishes that a security breach has been detected, taking further action to secure the computer network and/or any related network node, wherein the further action comprises one or more of the list of:
preventing one or more of the network nodes from being switched off;
switching on a firewall at one or more of the network nodes;
warning a user of one or more of the network nodes that signs of a security breach have been detected; and/or sending a software update to one or more of the network nodes.
10. A server comprising:
one or more processors, and
at least one non-transitory memory including computer program code, where the at least one non-transitory memory including the computer program code are configured with the one or more processors to cause the server configure to:
collect and align raw data from a plurality of network nodes, wherein dissimilar data types are aligned as input events;
filter the input events by discarding events and/or parts of events that are detected to be equal or similar to previously observed events or events and/or parts of events found to be redundant by using predetermined criteria;
separate processing of the input events into event aggregation and event enrichment processes, wherein
the event aggregation process comprises processing all the input events for generating aggregated events, and
the event enrichment process comprises processing only events passed by the filtering and the aggregated events from the event aggregation process; and
analyse the data received from the event enrichment process for generating a security related decision.
11. The server according to claim 10 , wherein filtering of the input events is based on a lookup logic defining processes for at least one of: generation of a lookup cache collecting information about already observed events, performing lookup queries to the lookup cache for checking which events have been observed already, determining essential elements of the passed through events required for the analysis, determining unnecessary elements of the passed through events to be culled.
12. The server according to claim 11 , where the at least one non-transitory memory including the computer program code are further configured with the one or more processors to cause the server to dynamically control the lookup logic on the basis of the analysis.
13. The server according to claim 11 , where the at least one non-transitory memory including the computer program code are further configured with the one or more processors to cause the server to:
determine for each input event a level of uniqueness on the basis of predetermined event elements, and using the determined level of uniqueness to decide whether an input event has been observed before in relation to the filtering.
14. The server according to claim 11 , wherein said event aggregation process comprises aggregating information about the input events to support the creation of different views according to specific types of entities related to the input events.
15. The server according to claim 11 , wherein said event aggregation process comprises creating aggregated events on the basis of predetermined event elements of the input events.
16. The server according to claim 11 , wherein said event enrichment process comprises extending the structure and context of events with previously collected data.
17. The server according to claim 11 , wherein said analysing the data comprises using at least one of the following processes for generating the decision:
predetermined rules, heuristics, machine learning models, fuzzy logic based models, statistical inference based model.
18. The server according to claim 11 , in case the generated security related decision establishes that a security breach has been detected, taking further action to secure the computer network and/or any related network node, wherein the further action comprises one or more of the list of:
preventing one or more of the network nodes from being switched off;
switching on a firewall at one or more of the network nodes;
warning a user of one or more of the network nodes that signs of a security breach have been detected; and/or sending a software update to one or more of the network nodes.
19. A non-transitory computer storage medium having stored thereon computer program code for-implementing:
collecting and aligning raw data from a plurality of network nodes, wherein dissimilar data types are aligned as input events;
filtering the input events by discarding events and/or parts of events that are detected to be equal or similar to previously observed events or events and/or parts of events found to be redundant by using predetermined criteria;
separating processing of the input events into event aggregation and event enrichment processes, wherein
the event aggregation process comprises processing all the input events for generating aggregated events, and
the event enrichment process comprises processing only events passed by the filtering and the aggregated events from the event aggregation process; and
analysing the data received from the event enrichment process for generating a security related decision.