Deduplication of encrypted data
View Patent ↗A mechanism is provided which allows to de-duplicate encrypted data such that the de-duplication ratio for encrypted data is similar to the de-duplication ration of the corresponding un-encrypted data and the purpose of encryption is not obfuscated, i.e. only the originator of the data (the client) can decrypt—and hence read—the data. This is achieved by interwoven the de-duplication algorithm with the encryption algorithm in a way that the data are encrypted with a key that is generated from the unencrypted data. Afterwards, that key is itself encrypted with an encryption key being private to a particular client. Due to the fact that the private key is not effecting the encrypted data stream, it can still be de-duplicated efficiently.
1. A computer implemented method for backing-up data within a computer environment, the method comprising:
storing a client file within a client repository on a client-device;
splitting, in a deterministic way the client file into chunks of data;
generating a client hash value for each respective chunk data from the client file thereby forming different client hash values;
encrypting each respective chunk of data using the respective client hash value thereby forming encrypted chunks of data;
encrypting the different client hash values using a client private key thereby forming encrypted different client hash values;
forwarding the encrypted chunks of data and the associated encrypted different client hash values to a server;
storing a client metadata at the server in a metadata repository associated to the client file, the client metadata mapping the encrypted chunks of data associated with the client file within the client repository and comprising the encrypted different client hash values;
generating a specific hash value at least partly generated from the chunks of data; and
using the specific hash value at least partly generated from the chunks of data for checking for duplicates of encrypted chunks of data, thereby forming a used hash value, in order to store only a single instance of identical encrypted chunk of data at the server, wherein a mapping of the used hash value to the encrypted chunk of data is stored into a hash map and the used hash value is referenced in the client metadata associated to the chunk of data.
2. The method according to claim 1 , wherein the used hash value for checking duplicates is generated by the client device using a deterministic combination of the respective hash values of the chunks of data from the client file, the chunks of data and the encrypted chunks of data, and the used hash value being forwarded to the server.
3. The method according to claim 1 , wherein the computer environment comprises several clients interconnected with the server via at least a communication link, the clients applying identical procedure for generation of hash value and for encryption.
4. The method according to claim 1 , wherein restoring the client file from the server comprises:
receiving a request from a requester client device for the client file from server thereby forming a requested client file;
looking-up the requested client file in the client metadata at the server;
using the encrypted different client hash values referenced in the client metadata associated to the chunks of data associated with the requested client file to map to the chunks of data;
sending to the requestor client device the encrypted different client hash values corresponding to the chunks of data associated with the requested client file stored in the metadata repository associated to the client file at the server;
decrypting by the requestor client device the encrypted different client hash values using the client private key of the requestor thereby forming a decrypted different client hash values;
sending to the requestor client device the encrypted chunks of data; and
decrypting by the requestor client device the encrypted chunks of data using the decrypted different client hash values.