Data repository and method for promoting network storage of data
A method by which objects that have content in common share storage in a data repository, the method comprising breaking an object into component data items at boundaries that depend upon the content of the object, determining digital fingerprints for the component data items, initiating processes to deposit a plurality of objects containing distinct occurrences of the identical component data item within their content, storing distinct information for some component data items of an object's content, but for other component data items relying on the identical component data item already stored, rather than storing a separate copy of the identical component data item.
1 . A method by which objects that have content in common share storage in a data repository, the method comprising:
breaking an object into component data items at boundaries that depend upon the content of the object;
determining digital fingerprints for the component data items;
initiating processes to deposit a plurality of objects containing distinct occurrences of the identical component data item within their content;
storing distinct information for some component data items of an object's content, but for other component data items relying on the identical component data item already stored, rather than storing a separate copy of the identical component data item.
2 . The method of claim 1 in which the boundaries are demarcated by structural elements within the object.
3 . The method of claim 2 in which the structural elements are attachment boundaries within an email message.
4 . The method of claim 1 in which objects are structured to segregate variable portions of their content into separate segments of the object.
5 . The method of claim 4 in which a user name is assigned to a variable at the beginning of a file, and only the variable name appears in subsequent portions of the file.
6 . The method of claim 1 in which the repeated occurrence of the identical data item is identified by a repository client by means of its fingerprint, and transmission of the identical data item as part of the deposit process is avoided.
7 . The method of claim 1 in which a client program communicates with the data repository using an NFS or CIFS file-sharing protocol.
8 . The method of claim 7 in which files and directories are objects within the data repository.
9 . The method of claim 1 in which data items are stored in the data repository at locations associated with their digital fingerprints.
10 . The method of claim 9 in which different locations correspond to different disk drives.
11 . The method of claim 1 in which the digital fingerprint of the data item is determined using a cryptographic hash.
12 . The method of claim 1 in which an object is deleted from the data repository which contains the identical data item as a component but another object remains in the data repository which has the identical data item as a component, and the identical data item is not deleted from the data repository.
13 . The method of claim 1 in which no objects remain in the data repository which contain the identical data item as a component, and the identical data item is deleted from the data repository.