Conserving file system with backup and validation
Disclosed herein is a hierarchical data storage and retrieval system implemented using a computed statistically unique signature for the content of given computer data as its basis. The data storage and retrieval system can be used to catalog computer information for easy indexing and retrieval. The signature is computed by any number of techniques so long as it is able to produce a statistically unique signature, one example being the SHA-1 algorithm. The system is able to reduce the amount of data that is stored and the time required for retrieving the data. Provide a way to move electronic data in an encrypted secure manner that requires the complete data set being moved for decryption of the data. Even if a portion of the data is compromised that portion would still remain secure. The data is encrypted and separated into two or more portions all of which would be needed to decrypt the data. Detailed information on various example embodiments of the inventions are provided in the Detailed Description below, and the inventions are defined by the appended claims.
1. A hierarchical data storage and retrieval system for reducing the amount of stored data and retrieval time, comprising: a processor; data and program storage accessible by said processor, said storage comprising one or more data storage devices; and computer readable instructions located to said program storage, wherein said instructions are configured to be executed by said processor to perform the functions of:
(i) receiving a request to store a piece of data, wherein the request includes a file name,
(ii) for the received request to store the piece of data, computing a signature for a content of the piece of data, wherein the signature is substantially statistically unique for the piece of data,
(iii) checking for an existence of the computed signature on the data storage and retrieval system that corresponds to an identical piece of data stored to said data storage, wherein the computed signature for the identical piece of data stored is identical to the computed signature for the piece of data in the request,
(iv) segmenting the computed signature into a plurality of signature segments used to catalog the content of the piece of data in a hierarchal directory,
(v) if in said checking the existence of the computed signature is not found, storing the piece of data associated with the received request to said data storage addressable by the computed signature, and further cataloging the piece of data associated with the received request by the file name and assigning to a counter a count corresponding to one instance of the piece of data associated with the received request,
(vi) if in said checking the existence of the computed signature is found, cataloging an existing piece of data stored to said data storage, if one exists, and incrementing the counter storing an instance count,
(vii) receiving a request to modify or delete a stored piece of data, the request including a file name,
(viii) for the received request to modify or delete the stored piece of data, identifying the stored piece of data cataloged by the received file name,
(ix) for the identified stored piece of data, decrementing the counter storing the instance count, and
(x) if in said decrementing the counter indicates zero instances of the identified stored piece of data, deleting the identified stored piece of data from the data storage and retrieval system and its corresponding catalog entry.
2. The hierarchical data storage and retrieval system of claim 1 , wherein the plurality of signature segments are used to generate subdirectories to catalog the content of the piece of data.
3. The hierarchical data storage and retrieval system of claim 1 , further comprising storing the content of the piece of data separately from the file name, a file path, a client name, client id, data and time information, or meta-data in the request.
4. The hierarchical data storage and retrieval system of claim 1 , wherein the content of the piece of data used to compute the computed signature is separate from the file name, a file path, a client name, client id, data and time information, and meta-data in the request.