Protocol-independent anomaly detection

1. A computer-implemented method for implementing protocol-independent anomaly detection within an industrial control system (ICS), comprising:

implementing a detection stage for the protocol-independent anomaly detection within the ICS, the ICS including unknown network protocols, including:

performing byte filtering using a byte filtering model based on at least one new network packet associated with the ICS, the byte filtering excluding zero-value entropies and comparatively high-value entropies from learning based on message entropy classification;

performing horizontal learning based on an output of the byte filtering to generate a horizontal model M, the horizontal learning comprising:

receiving as input a dataset S of network packets in a normal condition, a byte-level anomaly false positive threshold α, and a message-level anomaly false positive threshold β, wherein α, βϵ[0,1];

generating outputs including the horizontal model M, a horizontal filter for bytes β, internal byte-level horizontal anomaly thresholds for detection σ, and internal message-level anomaly thresholds for detection η based on the received input;

performing horizontal detection to determine whether a horizontal constraint anomaly exists in the at least one new network packet based on the byte filtering and the generated horizontal model, including analyzing constraints across different bytes of the at least one new network packet;

performing message clustering based on the horizontal detection to generate first cluster information; and

performing vertical detection to determine whether a vertical anomaly exists based on the first cluster information and a vertical model, including analyzing a temporal pattern of each byte of the at least one new network packet.

2. The method of claim 1 , where implementing the detection stage further includes:

recording the at least one new network packet for detection; and

preprocessing the at least one new network packet, including grouping the at least one new network packet with at least one existing network packet into at least one session by source IP address, source port number, destination IP address, destination port number, and protocol number the network packets.

3. The method of claim 2 , wherein preprocessing the at least one new network packet further includes determining new traffic based on a preprocessing model, and triggering an alert in response to the new traffic.

4. The method of claim 1 , wherein performing the byte filtering further includes finding a new violating pattern based on the byte filtering model, and triggering an alert in response to finding the new violating pattern.

5. The method of claim 1 , wherein performing the message clustering further includes determining a cluster of the at least one new network packet based on a probability distribution of the horizontal model.

6. The method of claim 1 , further comprising implementing a learning stage, including:

performing byte filtering based on one or more network packets to generate the byte filtering model;

performing message clustering based on the horizontal learning to generate second cluster information; and

performing vertical learning based on the second cluster information to generate a vertical model.

7. The method of claim 6 , wherein implementing the learning stage further includes:

recording the one or more network packets for learning; and

preprocessing the one or more network packets.

8. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method for implementing protocol-independent anomaly detection, the method performed by the computer comprising:

implementing a detection stage for the protocol-independent anomaly detection within the ICS, the ICS including unknown network protocols, including:

performing byte filtering using a byte filtering model based on at least one new network packet associated with the ICS, the byte filtering excluding zero-value entropies and comparatively high-value entropies from learning based on message entropy classification;

performing horizontal learning based on an output of the byte filtering to generate a horizontal model M, the horizontal learning comprising:

receiving as input a dataset S of network packets in a normal condition, a byte-level anomaly false positive threshold α, and a message-level anomaly false positive threshold β, where α, βϵ[0,1];

generating outputs including the horizontal model M, a horizontal filter for bytes B, internal byte-level horizontal anomaly thresholds for detection σ, and internal message-level anomaly thresholds for detection η based on the received input

performing horizontal detection to determine whether a horizontal constraint anomaly exists in the at least one new network packet based on the byte filtering and the generated horizontal model, including analyzing constraints across different bytes of the at least one new network packet;

performing message clustering based on the horizontal detection to generate first cluster information; and

performing vertical detection to determine whether a vertical anomaly exists based on the first cluster information and a vertical model, including analyzing a temporal pattern of each byte of the at least one new network packet.

9. The computer program product of claim 8 , where implementing the detection stage further includes:

recording the at least one new network packet for detection; and

preprocessing the at least one new network packet, including grouping the at least one new network packet with at least one existing network packet into at least one session by source IP address, source port number, destination IP address, destination port number, and protocol number of the network packets.

10. The computer program product of claim 9 , wherein preprocessing the at least one new network packet further includes determining new traffic based on a preprocessing model, and triggering an alert in response to the new traffic.

11. The computer program product of claim 8 , wherein performing the byte filtering further includes finding a new violating pattern based on the byte filtering model, and triggering an alert in response to finding the new violating pattern.

12. The computer program product of claim 8 , wherein performing the message clustering further includes determining a cluster of the at least one new network packet based on a probability distribution of the horizontal model.

13. The computer program product of claim 8 , wherein the method further includes implementing a learning stage, including:

performing byte filtering based on one or more network packets to generate the byte filtering model;

performing message clustering based on the horizontal learning to generate second cluster information; and

performing vertical learning based on the second cluster information.

14. The computer program product of claim 13 , wherein implementing the learning stage further includes:

recording the one or more network packets for learning; and

preprocessing the one or more network packets.

15. A system for implementing protocol-independent anomaly detection within an industrial control system (ICS), comprising:

a memory device for storing program code; and

at least one processor device operatively coupled to a memory device and configured to execute program code stored on the memory device to:

implement a detection stage for the protocol-independent anomaly detection within the ICS, the ICS including unknown network protocols, by:

performing byte filtering using a byte filtering model based on at least performing byte filtering using a byte filtering model based on at least one new network packet associated with the ICS, the byte filtering excluding zero-value entropies and comparatively high-value entropies from learning based on message entropy classification;

performing horizontal learning based on an output of byte filtering to generate a horizontal model M, the horizontal learning comprising:

receiving as input a dataset S of network packets in a normal condition, a byte-level anomaly false positive threshold α, and a message-level anomaly false positive threshold β, where α, βϵ[0,1];

generating outputs including the horizontal model M, a horizontal filter for bytes B, internal byte-level horizontal anomaly thresholds for detection σ, and internal message-level anomaly thresholds for detection η based on the received input;

performing horizontal detection to determine whether a horizontal constraint anomaly exists in the at least one new network packet based on the byte filtering and the generated horizontal model, including analyzing constraints across different bytes of the at least one new network packet;

performing message clustering based on the horizontal detection to generate first cluster information; and

performing vertical detection to determine whether a vertical anomaly exists based on the first cluster information and a vertical model, including analyzing a temporal pattern of each byte of the at least one new network packet.

16. The system of claim 15 , wherein the at least one processor device is further configured to implement the detection stage by:

recording the at least one new network packet for detection; and

preprocessing the at least one new network packet, including grouping the at least one new network packet with at least one existing network packet into at least one session by source IP address, source port number, destination IP address, destination port number, and protocol number of the network packets.

17. The system of claim 16 , wherein the at least one processor device is further configured to preprocess the at least one new network packet by determining new traffic based on a preprocessing model, and triggering an alert in response to the new traffic.

18. The system of claim 15 , wherein the at least one processor device is further configured to perform the byte filtering by finding a new violating pattern based on the byte filtering model, and triggering an alert in response to finding the new violating pattern.

19. The system of claim 15 , wherein the at least one processor device is further configured to perform the message clustering by determining a cluster of the at least one new network packet based on a probability distribution of the horizontal model.

20. The system of claim 15 , wherein the at least one processor device is further configured to implement a learning stage by:

recording one or more network packets for learning

preprocessing the one or more network packets;

performing byte filtering based on the preprocessing of the one or more network packets to generate the byte filtering model;

performing message clustering based on the horizontal learning to generate second cluster information; and

performing vertical learning based on the second cluster information.