Granted Patent B2

US 7,913,114 · App. 12/183,792 · Granted Mar 22, 2011

Abstract

Claims (59)

1. A method for repairing a corrupt data segment used by a de-duplication engine, said method comprising:

identifying a data segment, appearing in a first data stream and indexed by a de-duplication index, as a corrupt data segment;

searching for a duplicate data segment with which to replace said corrupt data segment, said duplicate data segment comprising an uncorrupted version of said data segment, and wherein said searching for the duplicate data segment comprises monitoring a de-duplication process to identify the duplicate data segment in one or more data streams subsequent to the first data stream; and

replacing said corrupt data segment with said duplicate data segment.

2. The method as recited in claim 1 , wherein said identifying a data segment indexed by a de-duplication index as a corrupt data segment comprises:

setting a corruption flag associated with said data segment.

3. The method as recited in claim 1 , wherein said identifying a data segment indexed by a de-duplication index as a corrupt data segment further comprises:

performing an integrity check on said data segment to determine if said data segment is corrupt; and

in response to said data segment failing said integrity check; storing information to indicate said data segment is corrupt.

4. The method as recited in claim 3 , wherein said performing an integrity check on said data segment comprises:

producing a re-generated hash value of said data segment; and

comparing said re-generated hash value of said data segment with a previously stored hash value of said data segment to determine if said data segment has changed.

5. The method as recited in claim 1 , wherein said searching for a duplicate data segment with which to replace said corrupt data segment comprises:

checking said de-duplication index to find said duplicate data segment.

6. A data de-duplication engine comprising:

a corrupt data identifier configured for identifying a data segment appearing in a first data stream and indexed by a de-duplication index as a corrupt data segment in response to a failed integrity check of the data segment;

a corrupt data replacement searcher configured for checking said de-duplication index and monitoring a data de-duplication process to find a duplicate data segment to said corrupt data segment either in said de-duplication index or during processing of data for de-duplication by identifying the duplicate data segment in one or more data streams subsequent to the first data stream, said duplicate data segment comprising an uncorrupted version of said data segment; and

a corrupt data replacer configured for replacing said corrupt data segment with said duplicate data segment.

7. The de-duplication engine of claim 6 , further comprising:

said de-duplication index, wherein said de-duplication index comprises an index of data segment identifiers associated with a plurality of data segments which have been identified, and wherein said data segment comprises one of said plurality of data segments indexed by said de-duplication index.

8. The de-duplication engine of claim 7 , wherein a data segment identifier of said data segment identifiers comprises a hash value.

9. The de-duplication engine of claim 6 , wherein said data segment comprises a data sub-block.

10. The de-duplication engine of claim 6 , wherein said data segment comprises a plurality of sub-blocks.

11. The de-duplication engine of claim 6 , wherein said corrupt data identifier comprises:

a data segment integrity checker configured for performing a scheduled integrity check on said data segment.

12. The de-duplication engine of claim 11 , wherein said data segment integrity checker comprises:

a hash generator for re-generating a hash value of said data segment; and

a hash comparator for comparing a re-generated hash value of said data segment with a previously stored hash value of said data segment to determine if said data segment has changed.

13. The de-duplication engine of claim 6 , wherein said corrupt data identifier comprises:

a settable corruption flag associated with said data segment and said de-duplication index.

14. A data storage system comprising:

at least one data storage mechanism; and

a de-duplication engine, said de-duplication engine comprising:

a corrupt data identifier configured for identifying a data segment appearing in a first data stream and indexed by a de-duplication index as a corrupt data segment in response to a failed integrity check of said data segment;

a corrupt data replacement searcher configured for checking said index and monitoring a data de-duplication process to find a duplicate data segment to said corrupt data segment either in said de-duplication index or during processing of data for de-duplication by identifying the duplicate data segment in one or more data streams subsequent to the first data stream, said duplicate data segment comprising an uncorrupted version of said data segment; and

a corrupt data replacer configured for replacing said corrupt data segment with said duplicate data segment.

15. The data storage system of claim 14 , wherein said at least one data storage mechanism comprises:

at least one hard disk drive.

16. The data storage system of claim 14 , wherein said at least one data storage mechanism comprises:

a redundant array of independent disks.

17. The data storage system of claim 14 , wherein said at least one data storage mechanism comprises:

a data cache.

18. The data storage system of claim 14 , wherein said de-duplication engine further comprises:

said de-duplication index, wherein said de-duplication index comprises an index of data segment identifiers associated with a plurality of data segments which have been identified, and wherein said data segment comprises one of said plurality of data segments indexed by said de-duplication index.

19. The data storage system of claim 14 , wherein said de-duplication engine further comprises:

a data segment integrity checker configured for performing a scheduled integrity check on said data segment.

20. A method for repairing a corrupt data segment used by a de-duplication engine, said method comprising:

identifying a data segment, indexed by a de-duplication index, as a corrupt data segment;

searching for a duplicate data segment with which to replace said corrupt data segment, said duplicate data segment comprising an uncorrupted version of said data segment; and

replacing said corrupt data segment with said duplicate data segment; and

wherein said searching for a duplicate data segment with which to replace said corrupt data segment comprises: checking said de-duplication index to find said duplicate data segment.

21. The method as recited in claim 20 , wherein said identifying a data segment indexed by a de-duplication index as a corrupt data segment further comprises:

performing an integrity check on said data segment to determine if said data segment is corrupt; and

in response to said data segment failing said integrity check; storing information to indicate said data segment is corrupt.

22. The method as recited in claim 21 , wherein said performing an integrity check on said data segment comprises:

producing a re-generated hash value of said data segment; and

comparing said re-generated hash value of said data segment with a previously stored hash value of said data segment to determine if said data segment has changed.

23. The method as recited in claim 20 , wherein said searching for a duplicate data segment with which to replace said corrupt data segment comprises:

monitoring a data de-duplication process to find said duplicate data segment.

Assignments (13)