System and method to enable deduplication engine to sustain operational continuity
A method for backing up data is provided. The method includes deduplicating a first plurality of data units during a backup operation through application of a first deduplication service and a fingerprint database. The method includes logging a second plurality of data units in a journal, during the backup operation. The logging is in response to a failure of the first deduplication service and wherein at least one method operation is executed through a processor.
1. A method for backing up data, comprising:
deduplicating a first plurality of data units during a backup operation through application of a first deduplication service and a fingerprint database; and
activating a second deduplication service in response to a failure mode of the first deduplication service to access fingerprint information from the fingerprint database, the second deduplication service including logging a second plurality of non-deduplicated data units in a journal to write fingerprints of the second plurality of non-duplicated data units into the journal during activation of the second deduplication service to form an auxiliary fingerprint database;
resuming operation of the first deduplication service in response to a recovery of the first deduplication service from the failure mode and wherein:
the second deduplication service is discontinued in response to the recovery of the first deduplication service from the failure mode;
initiating a recovery mode in response to the recovery in which the fingerprint database is updated with fingerprints from the auxiliary fingerprint database not found in the fingerprint database; and
at least one method of operation is executed through a processor.
2. The method of claim 1 , wherein the failure mode comprises the fingerprint database being unavailable and the second deduplication service is operable while the fingerprint database of the first deduplication service is unavailable.
3. The method of claim 1 , further comprising:
deduplicating the second plurality of non-deduplicated data units, via application of the second deduplication service and the journal.
4. The method of claim 1 , further comprising:
storing the second plurality of non-deduplicated data units in a backup storage, wherein the second plurality of non-deduplicated data units are non-deduplicated.
5. The method of claim 1 , further comprising:
storing the second plurality of non-deduplicated data units in a backup storage, wherein the second plurality of non-deduplicated data units is deduplicated via application of the second deduplication service and the journal.
6. The method of claim 1 , further comprising:
updating the fingerprint database, in the recovery mode, based upon log entries in the journal, the log entries being made as a result of logging the second plurality of non-deduplicated data units, the updating occurring responsive to the first deduplication service recovering from the failed mode to control the backup operation.
7. The method of claim 1 , further comprising:
processing the journal upon a resumption of the first deduplication service; and
adding fingerprints from the journal, which have no match in the fingerprint database, to the fingerprint database, as a result of the processing the journal.
8. The method of claim 1 , further comprising:
deleting redundant data units in a backup storage, as found through a processing of the journal, in a recovery of the first deduplication service.
9. A non-transient, tangible, computer-readable media having instructions thereupon which, when executed by a processor, cause the processor to:
derive, during a backup process, a first plurality of deduplicated data units from a first plurality of data units via a first deduplication process having a communication interface with a fingerprint database;
store the first plurality of deduplicated data units in a backup storage, during the backup operation;
store a second plurality of non-deduplicated data units in the backup storage during the backup operation, in response to a failure of the first deduplication process; and
store in a journal a history of storing the second plurality of non-deduplicated data units in the backup storage when a second deduplication process is active in which the history stored in the journal includes fingerprints of the second plurality of non-deduplicated data units that are organized into an auxiliary fingerprint database and, wherein storing the history of storing the second plurality of non-deduplicated data units is performed through application of the second deduplication process in response to the failure of the first deduplication process;
resume operation of the first deduplication service in response to a recovery of the first deduplication service from the failure mode; and
initiate a recovery mode in response to the recovery in which the fingerprint database is updated with fingerprints from the auxiliary fingerprint database not found in the fingerprint database.
10. The non-transient, tangible, computer-readable media of claim 9 , wherein the instructions further cause the processor to:
derive the second plurality of non-deduplicated data units from a third plurality of data units via a second deduplication process and an auxiliary fingerprint database, during the backup operation, wherein the journal includes the auxiliary fingerprint database.
11. The non-transient, tangible, computer-readable media of claim 9 , wherein the instructions further cause the processor to:
write first unmatched fingerprints of the first plurality of data units into the fingerprint database, wherein the first unmatched fingerprints are not matched in the fingerprint database;
add fingerprints from the journal to the fingerprint database, during a recovery of the first deduplication process, wherein the fingerprints added from the journal are not matched in the fingerprint database; and
execute the first deduplication process in a normal mode after the recovery of the first deduplication process.
12. A data backup system, comprising:
a backup storage;
a fingerprint database;
a first deduplication engine configured to execute a first deduplication process, the first deduplication process deduplicating first data units via comparison of fingerprints of the first data units to fingerprints in the fingerprint database, the first deduplication engine storing deduplicated first data units in the backup storage;
a journal;
a second deduplication engine configured to execute a second deduplication process, the second deduplication process storing second non-deduplicated data units in the backup storage, the second deduplication process logging fingerprints of the second non-deduplicated data units into the journal when the second deduplication engine is activated to form an auxiliary fingerprint database, the second deduplication process is idle while the first deduplication process deduplicates, the second deduplication process is active in storing data units during a condition of failure of the first deduplication process to deduplicate; and
wherein the first deduplication engine has a normal mode in which the first deduplication process is activated and a recovery mode in which the fingerprint database and the backup storage are updated utilizing the journal.
13. The data backup system of claim 12 , wherein:
the first deduplication engine is further configured to deduplicate the second non-deduplicated data units;
the second non-deduplicated data units are stored in the backup storage as non-deduplicated data units prior to deduplication by the first deduplication engine; and
the second non-deduplicated data units are stored in the backup storage as deduplicated data units after deduplication by the first deduplication engine.
14. The data backup system of claim 12 , wherein the second deduplication process being idle includes one from a set consisting of:
the second deduplication process polling the first deduplication engine to determine whether the condition of failure of the first deduplication process is true or false;
the second deduplication process being deactivated unless the first deduplication process fails a heartbeat check;
the first deduplication process and the second deduplication process being on separate threads in a multithreaded system, with the second deduplication process monitoring a first deduplication process thread;
the second deduplication process being activated by an interrupt, the interrupt issued upon the failure of the first deduplication process; and
the first deduplication process regularly resetting a timeout timer, the second deduplication process monitoring the timeout timer.
15. The data backup system of claim 12 , wherein the condition of failure of the first deduplication process to deduplicate includes one from a set consisting of:
the fingerprint database being unavailable;
the first deduplication process crashing;
the first deduplication engine ceasing operation;
a backup operation, under direction of the first deduplication engine, halting; and
corruption of the fingerprint database.
16. The data backup system of claim 12 , wherein:
the journal includes a log and the auxiliary fingerprint database; and
the second deduplication engine is configured to write a history of operation to the log and to write the fingerprints of the second data units to the auxiliary fingerprint database.
17. The data backup system of claim 12 , wherein the first deduplication engine is configured to:
look up journal entries in the journal after going back online following a failure of the first deduplication engine; and
update the fingerprint database, via application of the journal entries.