Intrusion detection using MDL compression

1. An intrusion masquerade detection method, comprising:

a computer applying a compression algorithm to user data to build user grammars associated with a user;

forming at least one model by storing the user grammars in a database;

applying the compression algorithm to at least one target block to calculate an estimated algorithmic minimum sufficient statistic;

searching a string of data from the at least one target block for phrases matching user grammars contained in the at least one model;

sorting the user grammars so that longest phrases among the user grammars are applied first to an unclassified string;

converting each matching phrase to a variable-length code value by replacing each matching phrase with a corresponding variable-length code value;

attributing a cost for phrases that are not found in the at least one model by quantifying a cost of explicitly representing symbols associated with those phrases;

determining a degree of fit between the at least one target block and the at least one model based on the cost; and

detecting an intrusion masquerade based on the degree of fit.

2. The intrusion detection method of claim 1 . wherein the searching is performed in real-time.

3. The intrusion detection method of claim 1 ,

wherein said the variable-length code is a Huffman code, and

the compression algorithm is a grammar-based compression algorithm that estimates Kolmogorov complexity and that forms compressive grammar based on Minimum Description Length (MDL) principles.

4. The intrusion detection method of claim 1 ,

wherein the forming the at least one model is performed using a steepest descent method, and

the at least one model comprises a healthy session model.

5. The intrusion detection method of claim 1 , further comprising:

outputting an indication of an intrusion masquerade.

6. The intrusion detection method of claim 1 ,

wherein the detecting the intrusion masquerade includes calculating an inverse compression ratio over a time period for user data, and comparing the calculated inverse compression ratio to at least one inverse compression ratio associated with a compressed data set of the at least one model, and

the detecting the intrusion masquerade is based on a difference between the calculated inverse compression ratio and the inverse compression ratio associated with the compressed data set, the difference exceeding a threshold.

7. A machine-implemented grammar inference engine for intrusion detection, comprising:

a pre-processor apparatus that receives input data and outputs filtered data;

a grammar generator apparatus coupled to the pre-processor apparatus and configured to generate grammars associated with a user by applying a compression algorithm to the filtered data, to form at least one model by storing the user grammars in a database, and to apply the compression algorithm to at least one target block to calculate an estimated algorithmic minimum sufficient statistic;

a grammar applicator apparatus that searches a string of data from the at least one target block for phrases matching user grammars contained in the at least one model, sorts the user grammars so that longest phrases among the user grammars are applied first to an unclassified string, replaces each matching phrase with a variable-length code value, and attributes a cost for phrases that are not found in the at least one model by quantifying a cost of explicitly representing symbols associated with those phrases; and

a classifier apparatus coupled to the grammar applicator apparatus and to a post-processor apparatus, the classifier apparatus receiving the cost from the grammar applicator apparatus and decision criteria from the post-processor apparatus, the classifier apparatus being configured to determine a degree of fit between the at least one target block and the at least one model based on the cost and the decision criteria, to detect an intrusion masquerade based on the degree of fit, and to output an indication of the detected intrusion masquerade,

wherein the post-processor apparatus assigns each portion of the input data to one of the models.

8. The grammar inference engine of claim 7 , further comprising:

a grammar database coupled to the grammar applicator apparatus and to the grammar generator apparatus; and

an input database coupled to an output of the pre-processor apparatus,

wherein the grammar applicator apparatus is configured to receive filtered data processed by the pre-processor apparatus from the input database.

9. The grammar inference engine of claim 7 , wherein the compression algorithm is a grammar-based compression algorithm that estimates Kolmogorov complexity and that forms compressive grammar based on Minimum Description Length (MDL) principles.

10. The grammar inference engine of claim 7 , wherein the pre-processor is further configured to apply a sliding window protocol to segment portions of the input data.

11. A non-transitory machine-readable medium upon which is embodied and stored a sequence of programmable instructions which, when executed by a processor, cause the processor to perform intrusion masquerade detection operations, comprising:

applying a compression algorithm to user data to build user grammars associated with a user;

forming at least one model by storing the user grammars in a database;

applying the compression algorithm to at least one target block to calculate an estimated algorithmic minimum sufficient statistic;

searching a string of data from the at least one target block for phrases matching user grammars contained in the at least one model;

attributing a cost for phrases that are not found in the at least one model by quantifying a cost of explicitly representing symbols associated with those phrases;

determining a degree of fit between the at least one target block and the at least one model based on the cost;

detecting an intrusion masquerade based on the degree of fit; and

outputting an indication of the detected intrusion masquerade.

12. The non-transitory machine-readable medium of claim 11 , wherein the operations further comprise:

sorting the user grammars so that longest phrases among the user grammars are applied first to an unclassified string.

13. The non-transitory machine-readable medium of claim 12 , wherein the operations further comprise:

replacing each matching phrase with a variable-length code value.

14. The non-transitory machine-readable medium of claim 13 , wherein the variable-length code is a Huffman code.

15. The non-transitory machine-readable medium of claim 11 , wherein the searching is performed in real-time.

16. The non-transitory machine-readable medium of claim 11 , wherein the forming the at least one model is performed using a steepest descent method.

17. The non-transitory machine-readable medium of claim 11 , wherein the at least one model comprises a healthy session model.

18. The non-transitory machine-readable medium of claim 11 ,

wherein the detecting the intrusion masquerade includes calculating an inverse compression ratio over a time period for user data, and comparing the calculated inverse compression ratio to at least one inverse compression ratio associated with a compressed data set of at the least one model, and

the detecting the intrusion masquerade event is based on a difference between the calculated inverse compression ratio and the inverse compression ratio associated with a compressed data set, the difference exceeding a threshold.

19. The non-transitory machine-readable medium of claim 11 , wherein the compression algorithm is a grammar-based compression algorithm that estimates Kolmogorov complexity and that forms compressive grammar based on Minimum Description Length (MDL) principles.

20. The non-transitory machine-readable medium of claim 11 , wherein the at least one target block comprises a plurality of information packets of an information system.