Network application security policy generation
Embodiments of the present invention generate network communication policies by applying machine learning to existing network communications, and without using information that labels such communications as healthy or unhealthy. The resulting policies may be used to validate communication between applications (or services) over a network.
1. A method comprising:
receiving flow objects from a plurality of computer systems, wherein each flow object is data based on communication associated with a first application at a first computer system that operates with a second application at a second computer system;
matching the flow objects with one another to determine match data, wherein two flow object match when they represent two corresponding flows at opposite ends of an application-to-application communication;
creating rule candidates within a network communication model generator, the rule candidates serving as an initial set of rules;
utilizing one of an evolutionary algorithm and a greedy algorithm to window down a set of rules from the initial set of rules; and
generating the network communication model based on the match data wherein the network communication model is or contains the winnowed set of rules resulting from the initial set of rules and is in a readable and modifiable form, and wherein the network communication model is configured to label a particular communication in the network as one of desired to be permitted or blocked.
2. The method of claim 1 , wherein the generating is performed utilizing unsupervised learning.
3. The method of claim 1 , wherein the generating includes
performing one or more passes over the match data;
growing rule trees in the network communication model based on evidence in the match data; and
terminating the multiple passes based on a stopping point.
4. A non-transitory computer-readable medium having computer program instructions stored thereon, the computer program instructions being executable by at least one computer processor in a remote server communicatively coupled to a network to perform steps of:
receiving flow objects from a plurality of computer systems, wherein each flow object is data based on communication associated with a first application at a first computer system that operates with a second application at a second computer system;
matching the flow objects with one another to determine match data, wherein two flow object match when they represent two corresponding flows at opposite ends of an application-to-application communication;
creating rule candidates within a network communication model generator, the rule candidates serving as an initial set of rules;
utilizing one of an evolutionary algorithm and a greedy algorithm to window down a set of rules from the initial set of rules; and
generating the network communication model based on the match data wherein the network communication model is or contains the winnowed set of rules resulting from the initial set of rules and is in a readable and modifiable form, and wherein the network communication model is configured to label a particular communication in the network as one of desired to be permitted or blocked.
5. The non-transitory computer-readable medium of claim 4 , wherein the steps further include:
providing the network communication model for labeling ongoing communications in the network based on the received flow objects.
6. The non-transitory computer-readable medium of claim 4 , wherein the generating is performed utilizing unsupervised learning.
7. The non-transitory computer-readable medium of claim 4 , wherein the generating includes:
performing one or more passes over the match data;
growing rule trees in the network communication model based on evidence in the match data; and
terminating the multiple passes based on a stopping point.
8. The non-transitory computer-readable medium of claim 7 , wherein the growing includes adding a new child node to the rule trees, with the new child node representing a feature-value pair representing sufficient evidence in the match data.
9. The non-transitory computer-readable medium of claim 7 , wherein the stopping point is based on one of a plurality of criteria including a certain number of matches, a certain number of the one or more passes, slowing of growth of the rules trees, and a certain size of the rule trees.
10. The non-transitory computer-readable medium of claim 7 , wherein the network communication model includes the rule trees subsequent to the terminating.
11. The non-transitory computer-readable medium of claim 4 , wherein the match data includes significantly more permitted communications than blocked communications.
12. The non-transitory computer-readable medium of claim 4 , wherein each flow object is generated locally at each computer system based on collected underlying network information.
13. A remote server comprising:
at least one computer processor; and
memory having computer program instructions, the computer program instructions being executable by the at least one computer processor communicatively coupled to a network to perform steps of:
receiving flow objects from a plurality of computer systems, wherein each flow object is data based on communication associated with a first application at a first computer system that operates with a second application at a second computer system;
matching the flow objects with one another to determine match data, wherein two flow object match when they represent two corresponding flows at opposite ends of an application-to-application communication;
creating rule candidates within a network communication model generator, the rule candidates serving as an initial set of rules;
utilizing one of an evolutionary algorithm and a greedy algorithm to window down a set of rules from the initial set of rules; and
generating the network communication model based on the match data wherein the network communication model is or contains the winnowed set of rules resulting from the initial set of rules and is in a readable and modifiable form, and wherein the network communication model is configured to label a particular communication in the network as one of desired to be permitted or blocked.
14. The remote server of claim 13 , wherein the steps further include:
providing the network communication model for labeling ongoing communications in the network based on the received flow objects.
15. The remote server of claim 13 , wherein the generating is performed utilizing unsupervised learning.
16. The remote server of claim 13 , wherein the generating includes:
performing one or more passes over the match data;
growing rule trees in the network communication model based on evidence in the match data; and
terminating the multiple passes based on a stopping point.
17. The remote server of claim 16 , wherein the growing includes adding a new child node to the rule trees, with the new child node representing a feature-value pair representing sufficient evidence in the match data.
18. The remote server of claim 16 , wherein the stopping point is based on one of a plurality of criteria including a certain number of matches, a certain number of the one or more passes, slowing of growth of the rules trees, and a certain size of the rule trees.
19. The remote server of claim 16 , wherein the network communication model includes the rule trees subsequent to the terminating.
20. The remote server of claim 13 , wherein the match data includes significantly more permitted communications than blocked communications.