Identifying history of modification within large collections of unstructured data
View Patent ↗A technique for efficient representation of dependencies between electronically-stored documents, such as in an enterprise data processing system. A document distribution path is developed as a directional graph that is a representation of the historic dependencies between documents, which is constructed in real time as documents are created. The system preferably maintains a lossy hierarchical representation of the documents indexed in such a way that allows for fast queries for similar but not necessarily equivalent documents. A distribution path, coupled with a document similarity service, can be used to provide a number of applications, such as a security solution that is capable of finding and restricting access to documents that contain information that is similar to other existing files that are known to contain sensitive information.
1. A method for maintaining a representation of a history of operations performed on document files in a data processing environment, the method comprising:
sensing access events that involve accessing one or more digital assets at a user client computer device, the step of sensing access events being carried out by a monitor process located within an operating system kernel of the user client computer device;
in response to sensing an access event involving an operation on an existing document file, determining a relationship descriptor that depends on the sensed access event;
in response to sensing an access event in which a new document file is created, comparing contents of the new document file with contents of existing document files contained in a database to measure a percentage of similar content between the contents of the new document file and the contents of the existing document files contained in the database to determine a relationship descriptor for the new document file, the relationship descriptor quantifying a degree by which at least one of the existing document files was modified to create the new document file; and
creating an entry in the representation that contains the relationship descriptor.
2. A method as in claim 1 wherein the representation includes other identification of the sensed access event, selected from a group including user identification, invoking process identification, network operation identification, and identification of storage media.
3. A method as in claim 2 wherein the storage media is removable.
4. A method as in claim 1 wherein the representation is represented as a graph having vertices that represent the document files and edges that represent a modification relationship between the document files.
5. A method as in claim 1 wherein the relationship descriptor identifies whether a source document file and the new document file are the same.
6. A method as in claim 1 wherein the relationship descriptor identifies a mechanism by which a source document file was modified to become the new document file.
7. A method as in claim 6 wherein the relationship descriptor is selected from a group including edit, merge, and copy.
8. A method as in claim 6 wherein the relationship descriptor further comprises identification of the mechanism by which the source document file was modified by monitoring a user's access to and/or modification events relevant to the source document file.
9. A method as in claim 1 wherein the representation is used to implement a data security application, and
if the measuring step determines that a similar document file exists, applying a security classification previously assigned to the similar document file to the new document file.
10. A method as in claim 9 wherein the monitor process limits user access to a document file according to its security classification.
11. A method as in claim 9 wherein the monitor process limits user control over a document file according to its security classification.
12. A method as in claim 9 wherein the security classification is applied to the new document file, in real time as the document file is initially stored in the system.
13. A method as in claim 1 wherein an initial modification relationship between at least one pair of files is determined by a degree of similarity between document files.
14. A method as in claim 13 wherein the modification relationship is further determined by an access time or modification time of at least one of the document files in the pair of document files.
15. A method as in claim 1 further including determining if the new document file was created from at least one of the existing document files based on the percentage of similar content between the contents of the new document file and the contents of the existing document files.
16. A method as in claim 15 wherein the relationship descriptor includes information relating to the creation of the new document file from the existing document files.
17. A method as in claim 1 further including transmitting the sensed access event to a server for processing.
18. A method as in claim 17 further including storing the representation in the server.