Protection of secret client data in a multiple client data deduplication environment

1. A method of deduplicating and protecting secret client data in a multiple client data deduplication and storage environment, comprising:

receiving in an enclave a block of secret plaintext data from a client, said enclave comprising a trusted execution environment which provides protected areas in an address space of an application executing in a processor in said enclave for processing confidential information intended to be accessed only by a designated recipient, and which enclave is inaccessible by unauthorized entities and other processes even those having administrative privileges;

combining in said enclave an initialization vector that is unique to said received block of secret plaintext data with said received block of secret plaintext data to individualize said received block of secret plaintext data;

encrypting the received block of secret plaintext data combined with its said unique initialization vector using a key that is calculated in said enclave by said application to produce a corresponding ciphertext block, said key being unique and individualized to said client, and storing said initialization vector and said key inside said enclave to prevent access to said initialization vector and key by entities outside of said enclave;

deduplicating said ciphertext block against previously stored ciphertext by using the block of received plaintext data that produced said ciphertext block; and

storing said deduplicated ciphertext block in the absence of previously stored ciphertext corresponding to said block of received plaintext data.

2. The method of claim 1 further comprising maintaining protected in said enclave from access encryption keys used to encrypt plaintext data to produce ciphertext data.

3. The method of claim 1 further comprising receiving a request from a requesting client for a block of plaintext data, verifying that the requesting client is authorized to receive the requested plaintext data, and, upon confirming authorization, retrieving from storage a block of ciphertext corresponding to the requested plaintext data, decrypting the retrieved block of ciphertext to produce the requested plaintext data, and returning to the requesting client the requested plaintext data.

4. The method of claim 1 , wherein said receiving comprises receiving from said client said plaintext data at said enclave via a secure transmission level secured communications channel, and wherein the multiple clients are in one or more deduplication domains in which data of said clients in one deduplication domain are deduplicated against data of other clients in said deduplication domain.

5. The method of claim 1 further comprising calculating a hash of data in said block of received plaintext data, and wherein said deduplicating comprises comparing said hash against hashes of previously received plaintext blocks, and, upon detecting a matching hash, further comparing metadata associated with said block of ciphertext corresponding to said block of received plaintext data to previous metadata associated with a previously received plaintext block having said matching hash, and upon said compared metadata being different, said storing comprises storing said block of ciphertext.

6. The method of claim 5 , wherein said comparing metadata comprises determining whether a key version of a key used to encrypt said previously received block of plaintext data is different from a key version used to encrypt said block of received plaintext data, and upon determining that the key versions are different, said storing comprises overwriting in storage a block of ciphertext corresponding to said previously received block of plaintext data with said block of ciphertext corresponding to said block of received plaintext data.

7. The method of claim 5 , wherein upon there being no matching hashes, storing the block of ciphertext corresponding to the received plaintext data block as non-duplicated data, and storing with said block of ciphertext said metadata associated with said block of ciphertext.

8. The method of claim 5 , wherein said metadata comprises an identifier of a key version and said initialization vector that were used to encrypt the corresponding block of received plaintext data to produce said block of ciphertext.

9. A method of deduplicating and protecting private client data in a remote multiple client data deduplication and storage environment, comprising:

encrypting private plaintext data of a client in an enclave to produce corresponding ciphertext, said enclave comprising a trusted execution environment providing protected areas for private client data in an address space of applications executing in said enclave, said address space being inaccessible except to designated entities and processes, said encrypting comprising combining an initialization vector that is unique to said private plaintext data with said private plaintext data to individualize said private plaintext data, and encrypting said private plaintext data combined with said unique initialization vector using a key that is calculated in said enclave, said key being unique and individualized to said client, and storing said initialization vector and said key in said enclave to prevent access to said initialization vector and key by entities outside of said enclave;

calculating a hash of said private plaintext data;

deduplicating said private plaintext data against previous ciphertext produced from other plaintext data by comparing said calculated hash to previously calculated hashes of said other plaintext data; and

upon determining that said private plaintext data is not duplicate data, storing said ciphertext corresponding to said deduplicated private plaintext data.

10. The method of claim 9 , wherein said client is a member of an ensemble of multiple clients that comprise a common deduplication domain, and said deduplicating comprises deduplicating said plaintext data against ciphertext produced from plaintext data of one or more of said multiple clients in said common deduplication domain.

11. The method of claim 9 , wherein said enclave has access to credentials for verifying clients which are authorized to send plaintext data to said enclave for encryption, deduplication and storage, and said method further comprises verifying a client before receiving, encrypting, deduplicating and storing plaintext data from such client.

12. The method of claim 9 , wherein said encrypting and deduplicating comprises encrypting and deduplicating compressed plaintext data from a client.

13. Computer readable non-transitory storage medium embodying executable instructions for controlling the operation of a processor to perform a method of deduplicating and protecting private client data in a remote data deduplication and storage environment, comprising:

initializing an enclave in said remote environment, said enclave comprising a trusted execution environment providing protected areas for private client data in an address space of applications executing in said enclave, said address space being inaccessible except to designated entities and processes;

encrypting in said enclave plaintext data of a client to produce corresponding ciphertext, said encrypting comprising combining an initialization vector that is unique to said private plaintext data with said private plaintext data to individualize said private plaintext data, and encrypting said private plaintext data combined with said unique initialization vector using a key that is calculated in said enclave, said key being unique and individualized to said client, and storing said initialization vector and said key in said enclave to prevent access to said initialization vector and key by entities outside of said enclave;

calculating a hash of said plaintext data;

deduplicating said corresponding ciphertext against previous ciphertext produced from other plaintext data by comparing said calculated hash to previously calculated hashes of said other plaintext data; and

upon determining that said plaintext data is not duplicate data, storing said corresponding ciphertext of said deduplicated private plaintext data.

14. The computer readable non-transitory storage medium of claim 13 , wherein said enclave has access to credentials for verifying clients which are authorized to send plaintext data to said enclave for encryption, deduplication and storage, and further comprising verifying a client before receiving, encrypting, deduplicating and storing plaintext data from such client.

15. The computer readable non-transitory storage medium of claim 13 further comprising receiving a request from a client for a block of plaintext data, verifying that the requesting client is authorized to receive the requested plaintext data, and, upon the client being authenticated, retrieving from storage a block of ciphertext corresponding to the requested plaintext data, decrypting the retrieved block of ciphertext to produce the requested plaintext data, and returning to the requesting client the requested plaintext data.

16. The computer readable non-transitory storage medium of claim 13 , wherein said deduplicating comprises comparing said calculated hash against said previously calculated hashes and, upon detecting a matching hash, determining whether the associated ciphertext that was produced with a different version of an encryption key than a key version used for said encrypting said plaintext data, and, if so, replacing said associated ciphertext and said previously stored metadata corresponding to said matching hash with said hash.

17. The computer readable non-transitory storage medium of claim 16 , wherein said client is a member of an ensemble of multiple clients that comprise a common deduplication domain, and said deduplicating comprises deduplicating said plaintext data against ciphertext produced from plaintext data of one or more of said multiple clients in said common deduplication domain using said hashes.