Selective shredding in a deduplication system

1. A method for making a target file impractical to be retrieved, comprising:

decrypting a directory manager file using a first directory manager file key, wherein the directory manager file includes an encryption key for a segment that is used when reconstructing a target file;

modifying the directory manager file by deleting a reference to the target file, wherein the reference includes a target file encryption key, and whereby retrieving the target file is made impractical by the deletion of the reference to the target file in the directory manager file; and

encrypting the modified directory manager file using a second directory manager file key.

2. A method as in claim 1 , further comprising receiving a directory manager metadata file.

3. A method as in claim 1 , further comprising receiving the first directory manager file key.

4. A method as in claim 1 wherein the encrypted modified directory manager file is stored.

5. A method as in claim 4 , wherein the encrypted modified directory manager file is stored in a hierarchical data structure.

6. A method as in claim 1 , wherein the target file encryption key comprises a hash of at least a portion of a content of the target file.

7. A method as in claim 6 , wherein the hash comprises one or more of the following: an HMAC keyed hash function or a cryptographically secure hash function.

8. A method as in claim 1 , wherein the target file is stored as one or more segments in a hierarchical data structure.

9. A method as in claim 8 , wherein the one or more segments are stored in a plurality of tiers, wherein an encryption key for one of the one or more segments in a lower tier are stored in as an encrypted data in a segment in a next higher tier.

10. A method as in claim 8 , wherein the one or more segments are stored in containers, wherein each container includes one or more data segments and metadata.

11. A method as in claim 10 , wherein the one or more segments are encrypted.

12. A method as in claim 10 , wherein the metadata comprises fingerprint data.

13. A method as in claim 12 , wherein the metadata comprising fingerprint data are unencrypted enabling deduplication of data segments identified by the metadata without decrypting the data segments.

14. A method as in claim 10 , wherein the metadata is based at least in part on the target file encryption key.

15. A method as in claim 10 , wherein the one or more data segments are locatable based on the metadata.

16. A method as in claim 10 , wherein the metadata is computed by a one-way function of the target file encryption key.

17. A method as in claim 16 , wherein the one-way function comprises one of the following: exponential modulo p function, the RSA function, the Rabin function, or a one-way function that has been constructed from a block cipher.

18. A method as in claim 17 , wherein the block cipher comprises one of the following: DES or AES.

19. A method as in claim 1 , further comprising deleting the first directory manager file key.

20. A method as in claim 1 , further comprising receiving the second directory manager file key.

21. A method as in claim 1 , further comprising deleting the target file referred to by the reference.

22. A method as in claim 1 , further comprising deleting the metadata corresponding to the target file referred to by the reference.

23. A computer program product for making a target file impractical to be retrieved, the computer program product being embodied in a non-transitory computer readable medium and comprising computer instructions for:

decrypting a directory manager file using a first directory manager file key, wherein the directory manager file includes an encryption key for a segment that is used when reconstructing a target file;

modifying the directory manager file by deleting a reference to the target file, wherein the reference includes a target file encryption key, and whereby retrieving the target file is made impractical by the deletion of the reference to the target file in the directory manager file; and

encrypting the modified directory manager file using a second directory manager file key.

24. A system for making a target file impractical to be retrieved, comprising:

a processor configured to:

decrypt a directory manager file using a first directory manager file key, wherein the directory manager file includes an encryption key for a segment that is used when reconstructing a target file;

modify a directory manager file by deleting a reference to the target file, wherein the reference includes a target file encryption key, and whereby retrieving the target file is made impractical by the deletion of the reference to the target file in the directory manager file;

encrypting the modified directory manager file using a second directory manager file key; and

a memory coupled to the processor and configured to provide the processor with instructions.

25. A system as in claim 24 , wherein the processor is configured to receive a directory manager metadata file.

26. A system as in claim 24 , wherein the processor is configured to receive the first directory manager file key.

27. A system as in claim 24 , wherein the encrypted modified directory manager file is stored.

28. A system as in claim 27 , wherein the encrypted modified directory manager file is stored in a hierarchical data structure.

29. A system as in claim 24 , wherein the target file encryption key comprises a hash of at least a portion of a content of the target file.

30. A system as in claim 29 , wherein the hash comprises one or more of the following: an HMAC keyed hash function or a cryptographically secure hash function.

31. A system as in claim 24 , wherein the target file is stored as one or more segments in a hierarchical data structure.

32. A system as in claim 31 , wherein the one or more segments are stored in a plurality of tiers, wherein an encryption key for one of the one or more segments in a lower tier are stored in as an encrypted data in a segment in a next higher tier.

33. A system as in claim 31 , wherein the one or more segments are stored in containers, wherein each container includes one or more data segments and metadata.

34. A system as in claim 33 , wherein the one or more segments are encrypted.

35. A system as in claim 33 , wherein the metadata comprises fingerprint data.

36. A system as in claim 35 , wherein the metadata comprising fingerprint data are unencrypted enabling deduplication of data segments identified by the metadata without decrypting the data segments.

37. A system as in claim 33 , wherein the metadata is based at least in part on the target file encryption key.

38. A system as in claim 33 , wherein the one or more data segments are locatable based on the metadata.

39. A system as in claim 33 , wherein the metadata is computed by a one-way function of the target file encryption key.

40. A system as in claim 39 , wherein the one-way function comprises one of the following: exponential modulo p function, the RSA function, the Rabin function, or a one-way function that has been constructed from a block cipher.

41. A system as in claim 40 , wherein the block cipher comprises one of the following: DES or AES.

42. A system as in claim 24 , wherein the processor is further configured to delete the first directory manager file key.

43. A system as in claim 24 , wherein the processor is further configured to receive the second directory manager file key.

44. A system as in claim 24 , wherein the processor is further configured to delete the target file referred to by the reference.

45. A system as in claim 24 , wherein the processor is further configured to delete the metadata corresponding to the target file referred to by the reference.