Method and apparatus for network forensics compression and storage

1. A method for network forensics compression and storage, comprising the following steps:

Step S 1 , extracting high level network service operation data;

Step S 2 , segmenting the network service operation data to obtain a plurality of data blocks;

Step S 3 , calculating a hash value for each data block;

Step S 4 , determining whether the hash value already exists in the hash mapping table or not, if the hash value already exists in the hash mapping table, saving the mapping relationship between the data block and the corresponding hash value in the data mapping table;

if the hash value does not exist in the hash mapping table, saving the mapping relationship between the hash value and the corresponding original data content in the hash mapping table and then saving the mapping relationship between the data block and the corresponding hash value in the data mapping table,

wherein the network service operation data comprises network service operation detailed information reflecting user's network operation details, and

wherein the step of segmenting the network service operation data in step S 2 comprises the following steps:

Step S 21 , segmenting the network service operation data according to a fixed length;

Step S 23 , moving the segmentation boundary of the network service operation data back and forth according to a predetermined byte range till a byte string with the highest occurrence probability, then replacing the segmentation boundary with starting position of the byte string.

2. The method according to claim 1 , wherein before step S 23 , it further comprises:

Step S 22 , loading a byte string dictionary table, which contains occurrence probability of the byte string.

3. The method according to claim 2 , wherein the byte string dictionary table is a fixed-length byte string dictionary table trained based on corpora.

4. An apparatus for network forensic compression and storage, comprising the following modules stored as computer program instructions in non-transitory memory:

a data extracting module in non-transitory memory for extracting high level network service operation data;

a data segmenting module in non-transitory memory for segmenting the network service operation data extracted by the data extraction module to obtain a plurality of data blocks;

a data mapping module in non-transitory memory for calculating a hash value for each data block obtained by the data segmenting module;

a data storage module in non-transitory memory for determining whether the hash value already exists in the hash mapping table or not, if the hash value already exists in the hash mapping table, saving the mapping relationship between the data block and the corresponding hash value in the data mapping table; if the hash value does not exist in the hash mapping table, saving the mapping relationship between the hash value and the corresponding original data content in the hash mapping table and then saving the mapping relationship between the data block and the corresponding hash value in the data mapping table,

wherein the network service operation data comprises network service operation detailed information reflecting user's network operation details,

wherein the data segmenting module further comprises:

a segmenting unit for segmenting the network service operation data according to a fixed length;

an adjusting unit for moving the segmentation boundary of the network service operation data back and forth according to a predetermined byte range till a byte string with the highest occurrence probability, then replacing the segmentation boundary with starting position of the byte string.

5. The apparatus according to claim 4 , wherein the data segmenting module further comprises:

a dictionary table loading unit for loading a byte string dictionary table, which contains the occurrence probability of the byte string.

6. The apparatus according to claim 5 , wherein the byte string dictionary table is a fixed-length byte string dictionary table trained based on corpora.

7. A computer readable storage medium having stored thereon computer program instructions in non-transitory memory for performing the following steps on a computer:

Step S 1 , extracting high level network service operation data;

Step S 2 , segmenting the network service operation data to obtain a plurality of data blocks;

Step S 3 , calculating a hash value for each data block;

Step S 4 , determining whether the hash value already exists in the hash mapping table or not, if the hash value already exists in the hash mapping table, saving the mapping relationship between the data block and the corresponding hash value in the data mapping table; if the hash value does not exist in the hash mapping table, saving the mapping relationship between the hash value and the corresponding original data content in the hash mapping table and then saving the mapping relationship between the data block and the corresponding hash value in the data mapping table,

wherein the network service operation data comprises network service operation detailed information reflecting user's network operation details, and

wherein the step of segmenting the network service operation data in step S 2 comprises the following steps:

Step S 21 , segmenting the network service operation data according to a fixed length;

Step S 23 , moving the segmentation boundary of the network service operation data back and forth according to a predetermined byte range till a byte string with the highest occurrence probability, then replacing the segmentation boundary with starting position of the byte string.

8. The storage medium according to claim 7 , wherein before step S 23 , it further comprises:

Step S 22 , loading a byte string dictionary table, which contains occurrence probability of the byte string.

9. The storage medium according to claim 8 , wherein the byte string dictionary table is a fixed-length byte string dictionary table trained based on corpora.