High-performance network content analysis platform
One implementation of a method reassembles complete client-server conversation streams, applies decoders and/or decompressors, and analyzes the resulting data stream using multi-dimensional content profiling and/or weighted keyword-in-context. The method may detect the extrusion of the data, for example, even if the data has been modified from its original form and/or document type. The decoders may also uncover hidden transport mechanisms such as, for example, e-mail attachments. The method may further detect unauthorized (e.g., rogue) encrypted sessions and stop data transfers deemed malicious. The method allows, for example, for building 2 Gbps (Full-Duplex)-capable extrusion prevention machines.
1. A method comprising:
receiving network data;
processing the network data at one or more decoders to create input data for applying at least multi-dimensional content profiling;
preventing, by a processor, through the network data, leaks of information by at least applying the multi-dimensional content profiling; and
wherein the multi-dimensional content profiling comprises:
loading one or more profiles, wherein the one or more profiles each comprise an expected set of statistical characteristics of data;
continuously receiving the input data from the one or more decoders;
determining a probabilistic measure of membership of the input data relative to the one or more profiles;
comparing the probabilistic measure with a threshold requirement for each of the one or more profiles; and
preventing leaks of the information if the probabilistic measure meets the threshold requirement.
2. The method of claim 1 , wherein the information includes a digital asset.
3. The method of claim 1 , wherein the multi-dimensional content profiling takes into account the structure of the information.
4. The method of claim 1 , wherein processing the network data at the decoder chain comprises extracting data by removing one or more layers of content encoding selected from the group consisting of common compression, aggregation, file formats, encoding schemas, and combinations thereof.
5. The method of claim 1 , further comprising creating the profile by:
loading positive training sets of documents;
representing each document from the positive training sets of documents as a point in multi-dimensional space;
separating the individual points in the multi-dimensional space with a set of hyperplanes wherein the set of hyperplanes effectively separate the multi-dimensional space into regions representing the positive training sets of documents; and
converting the set of hyperplanes into the profile.
6. The method of claim 5 , further comprising creating the profile by:
loading negative training sets of documents;
representing each document from the negative training sets of documents as a point in multi-dimensional space;
separating the individual points in the multi-dimensional space with a set of hyperplanes wherein the set of hyperplanes effectively separate the multi-dimensional space into regions representing the negative training sets of documents; and
converting the set of hyperplanes into the profile.
7. The method of claim 1 , wherein determining the probabilistic measure comprises updating one or more counters in a predetermined order, calculating values of output dimensions based on the one or more counters, and calculating an output score based on the output dimensions wherein the output score represents the probabilistic measure.
8. The method of claim 1 , wherein the preventing operates in real-time.
9. The method of claim 1 , further comprising terminating sessions with leaks of information before the network data is fully transferred.
10. The method of claim 1 , further comprising preventing, through the network data, leaks of information by also applying keyword scanning.
11. A machine-readable storage medium having encoded information, which when read and executed by a machine causes a method comprising:
receiving network data;
processing the network data at one or more decoders to create input data for applying at least multi-dimensional content profiling;
preventing, through the network data, leaks of information by at least applying multi-dimensional content profiling; and
wherein the multi-dimensional content profiling comprises:
loading one or more profiles, wherein the one or more profiles each comprise an expected set of statistical characteristics of data;
continuously receiving the input data from the one or more decoders;
determining a probabilistic measure of membership of the input data relative to the one or more profiles;
comparing the probabilistic measure with a threshold requirement for each of the one or more profiles; and
preventing leaks of the information if the probabilistic measure meets the threshold requirement.
12. The machine-readable storage medium of claim 11 , wherein the profile is created by:
loading positive training sets of documents;
representing each document from the positive training sets of documents as a point in multi-dimensional space;
separating the individual points in the multi-dimensional space with a set of hyperplanes wherein the set of hyperplanes effectively separate the multi-dimensional space into regions representing the positive training sets of documents; and
converting the set of hyperplanes into the profile.
13. The machine-readable storage medium of claim 12 , wherein the profile is created by:
loading negative training sets of documents;
representing each document from the negative training sets of documents as a point in multi-dimensional space;
separating the individual points in the multi-dimensional space with a set of hyperplanes wherein the set of hyperplanes effectively separate the multi-dimensional space into regions representing the negative training sets of documents; and
converting the set of hyperplanes into the profile.
14. The machine-readable storage medium of claim 11 , further comprising receiving the network data at a decoder chain prior to implementing the multi-dimensional content profiling, wherein the decoder chain extracts data by removing one or more layers of content decoding selected from the group consisting of common compression, aggregation, file formats, encoding schemas, and combinations thereof
15. The machine-readable storage medium of claim 11 , wherein the multi-dimensional content profiling further comprises establishing a connection with an alert module prior to sending the reactive measure.
16. The machine-readable storage medium of claim 11 , wherein the calculating the set of output dimensions comprises determining one or more values for each counter and combining the one or more values for each counter to create the set of output dimensions.
17. An apparatus comprising:
a receiver to receive network data;
a processor, coupled to the receiver, to prevent, through the network data, leaks of information by at least applying multi-dimensional content profiling, wherein the processor processes the network data at one or more decoders to create input data for applying at least the multi-dimensional content profiling;
the multi-dimensional content profiling comprising:
loading one or more profiles, wherein the one or more profiles each comprise an expected set of statistical characteristics of data;
continuously receiving the input data from the one or more decoders;
determining a probabilistic measure of membership of the input data relative to the one or more profiles;
comparing the probabilistic measure with a threshold requirement for each of the one or more profiles; and
preventing leaks of the information if the probabilistic measure meets the threshold requirement.
18. The apparatus of claim 17 , wherein the calculating an output score is performed for each of the one or more profiles.