Data de-duplication for information storage systems

1. A method for performing de-duplication operations in a data storage system, comprising:

coordinating with a kernel module of the data storage system to identify a unit of data, wherein the unit of data is part of a logical block of data stored in physical storage of the data storage system;

reading the identified unit of data;

calculating a signature of the identified unit of data;

determining whether the identified unit of data is a candidate for de-duplication by searching a de-duplication data structure for the signature of the identified unit of data; and

performing block-level de-duplication of the logical block of data by de-duplicating the candidate for de-duplication, wherein the coordination with the kernel module of the data storage system further comprises consulting a volume table associated with the logical block of data to identify the unit of data, and wherein the identified unit of data is a next logical chunk in the logical block to read for de-duplication operations.

2. The method of claim 1 , wherein coordinating with a kernel module of the data storage system further comprises determining a unit of data to read using Information Lifecycle Management (ILM) information.

3. The method of claim 2 , wherein the ILM information comprises a frequency of access for the identified unit of data or a time of last access for the identified unit of data.

4. The method of claim 1 , wherein coordinating with a kernel module of the data storage system further comprises determining a unit of data to read using First Write Bitmap (FWB) information.

5. The method of claim 1 , wherein coordinating with a kernel module of the data storage system further comprises determining a unit of data to read using re-write bitmap information.

6. The method of claim 1 , further comprising, upon determining that the signature of the identified unit of data is not stored in the de-duplication data structure, adding a new entry to the de-duplication data structure.

7. The method of claim 1 , further comprising, upon determining that the signature of the identified unit of data is stored in the de-duplication data structure, queueing the identified unit of data for de-duplication.

8. The method of claim 1 , wherein the de-duplication data structure comprises a plurality of entries, each entry corresponding to a respective unit of data and comprising a respective signature of the respective unit of data.

9. The method of claim 8 , wherein the de-duplication data structure comprises a B+ tree.

10. The method of claim 8 , wherein the de-duplication data structure comprises a plurality of B+ trees.

11. A non-transitory computer-readable storage medium having computer-executable instructions stored thereon for performing de-duplication operations that, when executed by a storage system computer, cause the storage system computer to:

coordinate with a kernel module of a data storage system to identify a unit of data, wherein the unit of data is part of a logical block of data stored in physical storage of the data storage system;

read the identified unit of data;

calculate a signature of the identified unit of data;

determine whether the identified unit of data is a candidate for de-duplication by searching a de-duplication data structure for the signature of the identified unit of data; and

perform block-level de-duplication of the logical block of data by de-duplicating the candidate for de-duplication, wherein the coordination with the kernel module of the data storage system further comprises consulting a volume table associated with the logical block of data to identify the unit of data, and wherein the identified unit of data is a next logical chunk in the logical block to read for de-duplication operations.

12. The non-transitory computer-readable storage medium of claim 11 , wherein coordinating with a kernel module of the data storage system further comprises determining a unit of data to read using Information Lifecycle Management (ILM) information.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the ILM information comprises a frequency of access for the identified unit of data or a time of last access for the identified unit of data.

14. The non-transitory computer-readable storage medium of claim 11 , wherein coordinating with a kernel module of the data storage system further comprises determining a unit of data to read using First Write Bitmap (FWB) information.

15. The non-transitory computer-readable storage medium of claim 11 , wherein coordinating with a kernel module of the data storage system further comprises determining a unit of data to read using re-write bitmap information.

16. The non-transitory computer-readable storage medium of claim 11 , having further computer-executable instructions stored thereon that, when executed by the storage system computer, cause the storage system computer to, upon determining that the signature of the identified unit of data is not stored in the de-duplication data structure, add a new entry to the de-duplication data structure.

17. The non-transitory computer-readable storage medium of claim 11 , having further computer-executable instructions stored thereon that, when executed by the storage system computer, cause the storage system computer to, upon determining that the signature of the identified unit of data is stored in the de-duplication data structure, queue the identified unit of data for de-duplication.

18. The non-transitory computer-readable storage medium of claim 11 , wherein the de-duplication data structure comprises a plurality of entries, each entry corresponding to a respective unit of data and comprising a respective signature of the respective unit of data.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the de-duplication data structure comprises a B+ tree.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the de-duplication data structure comprises a plurality of B+ trees.