Data loss monitoring of partial data streams

1. A computer-implemented method comprising:

identifying partial data streams containing segments lost while capturing network traffic at a network computing device, each partial data stream corresponding to a session;

determining characteristics of the partial data streams, wherein determining characteristics of the partial data streams comprises performing a protocol analysis for data stream elements of a partial data stream by determining a type for each data stream element, parsing each data stream element based on the type to extract metadata, and storing the metadata in a metadata store;

padding content portions of the lost segments in the partial data streams to generate padded partial data streams; and

scanning the padded partial data streams for sensitive information according to at least one data loss prevention (DLP) policy, wherein scanning the padded partial data streams comprises using signatures of the sensitive information to determine whether content portions of the padded partial data streams contain the sensitive information.

2. The computer-implemented method of claim 1 , wherein identifying partial data streams comprises:

capturing the network traffic containing a plurality of segments at the network computing device;

creating a data stream each time a segment is identified as a first segment in the data stream based on a transport layer header of the segment;

associating the data stream with a session;

identifying segments that belong to the data stream using transport layer headers of the segments;

identifying segments that are missing from the data stream;

placing the missing segments on a lost segment queue; and

upon receiving an end of stream indication or upon an expiration of a predefined time interval, marking the segments remaining on the lost segment queue as lost segments.

3. The computer-implemented method of claim 2 , further comprising:

identifying a data segment associated with a connection tuple for which a data stream has not been created;

determining that a first segment associated with the connection tuple has been lost;

creating a partial data stream based on the connection tuple;

determining whether the partial data stream belongs to a previously created session;

if the partial data stream does not belong to the previously created session, associating the partial data stream with a new session; and

if the partial data stream belongs to the previously created session, migrating the partial data stream to the previously created session.

4. The computer-implemented method of claim 1 , wherein determining characteristics of the partial data streams further comprises:

after identifying a lost segment, inserting a begin marker and an end marker at a location of the lost segment in the partial data stream; and

recording the location of the lost segment in the metadata store.

5. The computer-implemented method of claim 4 , wherein determining the type for each data stream element comprises:

if the type of the data stream element is known, identifying a parser corresponding to the type of the data stream element; and

if the type of the data stream element is not known,

applying a plurality of signatures to the data stream element until finding a matching signature, each of the plurality of signatures corresponding to a particular type,

identifying the type of the data stream element using the matching signature, and

identifying a parser corresponding to the type of the data stream element.

6. The computer-implemented method of claim 4 , wherein applying a plurality of signatures to the data stream element comprises:

aligning the data stream element to start at a first possible header tag;

applying the plurality of signatures at each possible header tag; and

identifying a header type using the matching signature.

7. The computer-implemented method of claim 6 , wherein:

the header type is an application layer protocol header; and

the metadata extracted from the application layer protocol header comprises at least one of sender information, recipient information, a data type and a data length.

8. The computer-implemented method of claim 6 , wherein:

the header type is a presentation layer header; and

the metadata extracted from the presentation layer header comprises a data format.

9. The computer-implemented method of claim 6 , wherein the matching signature uses a header format and a plurality of tags associated with the header type, the plurality of tags comprising a plurality of primary tags and a plurality of additional tags.

10. The computer-implemented method of claim 5 , wherein identifying the type of the data stream element further comprises:

utilizing metadata collected when identifying a type of a prior data stream element of the data stream.

11. The computer-implemented method of claim 4 , wherein padding the content portion of lost segments in the partial data stream to generate padded partial data streams comprises:

identifying a location of each content portion present in the partial data stream;

identifying a type of each content portion present in the partial data stream;

tagging the content portion of each lost segment as invalid;

determining whether a transformation of any content portion present in the partial data stream is required; and

if the transformation of any content portion present in the partial data stream is required, identifying a type of the required transformation, and transforming the content portion using the identified type of the required transformation.

12. The computer-implemented method of claim 1 , wherein scanning the padded partial data streams for sensitive information according to the at least one DLP policy comprises:

identifying portions of content that are missing; and

ignoring the portions of content that are missing when scanning the content for the sensitive information using signatures of the sensitive information.

13. The computer-implemented method of claim 12 , wherein the content is scanned using k-gram signatures.

14. The computer-implemented method of claim 12 , further comprising:

upon detecting the sensitive information in the padded partial data stream, reporting a violation of the DLP policy and a percentage of missing content in the padded partial data stream.

15. A non-transitory computer readable storage medium that provides instructions, which when executed on a computer system cause the computer system to perform a method comprising:

identifying partial data streams containing segments lost while capturing network traffic at a network computing device, each partial data stream corresponding to a session;

determining characteristics of the partial data streams, wherein determining characteristics of the partial data streams comprises performing a protocol analysis for data stream elements of a partial data stream by determining a type for each data stream element, parsing each data stream element based on the type to extract metadata, and storing the metadata in a metadata store;

padding content portions of the lost segments in the partial data streams to generate padded partial data streams; and

scanning the padded partial data streams for sensitive information according to at least one data loss prevention (DLP) policy, wherein scanning the padded partial data streams comprises using signatures of the sensitive information to determine whether content portions of the padded partial data streams contain the sensitive information.

16. The non-transitory computer readable storage medium of claim 15 , wherein the method further comprises:

after identifying a lost segment, inserting a begin marker and an end marker at a location of the lost segment in the partial data stream; and

recording the location of the lost segment in the metadata store.

17. A network device comprising:

a memory;

a processing device, coupled to the memory; and

a partial data stream subsystem, executed from the memory by the processing device, to:

identify partial data streams containing segments lost while capturing network traffic at a network computing device, each partial data stream corresponding to a session;

determine characteristics of the partial data streams, wherein determining characteristics of the partial data streams comprises performing a protocol analysis for data stream elements of a partial data stream by determining a type for each data stream element, parsing each data stream element based on the type to extract metadata, and storing the metadata in a metadata store;

pad content portions of the lost segments in the partial data streams to generate padded partial data streams; and

scan the padded partial data streams for sensitive information according to at least one data loss prevention (DLP) policy, wherein scanning the padded partial data streams comprises using signatures of the sensitive information to determine whether content portions of the padded partial data streams contain the sensitive information.

18. The network device of claim 17 , wherein the partial data stream subsystem comprises:

a protocol analyzer to perform the protocol analysis for data stream elements of a partial data stream using a plurality of signatures;

the metadata store to store metadata extracted during the protocol analysis;

a content manager to identify a location and a type of each content portion present in the partial data stream, to tag the content portion of each lost segment as invalid, and to transform any content portion present in the partial data stream that requires transformation; and

a DLP scanner to identify portions of content that are missing, to ignore the portions of content that are missing when scanning the content for the sensitive information using signatures of the sensitive information, and to report a violation of the DLP policy and a percentage of missing content in the padded partial data stream upon detecting the sensitive information in a padded partial data stream.