Systems and methods of backup and recovery of journaling systems
In part, the disclosure relates to a backup and restoration system for a transactional log based journaling application. The system includes a transactional log backup process executing on one or more computing devices; an archive stored in non-transitory computer readable memory; and a binary difference file generator in electronic communication with the archive and responsive to instructions from the transactional log backup process. In one embodiment, the binary difference file generator includes a backup driver in electrical communication with and responsive to communication signals from the transactional backup process.
1. A method for backup and restoration of a transactional log-based journaling application comprising a database, the method comprising:
running the database on a first computing device;
receiving n transactional logs from the database;
saving the n transactional logs into an archive;
tracking changes in the database using a file tracker driver since a previous backup;
minimizing the number of transactional logs replayed into database by creating one or more binary difference files (DIFF);
creating, using the binary difference file generator and changes tracked using the file tracker driver, the one or more binary difference files (DIFF) of the database from one or more of the n transactional logs;
saving the one or more DIFF;
selecting a restore type that is initiated based on a selected point in time and using a first set of DIFF for a first restore process or
selecting a restore type that is initiated based on a fault or other point of failure and using a second set of DIFF for a second restore process; and
reducing processing load on the first computing device by restoring one or more transactions to the database using the first set of DIFF or the second set of DIFF instead of the transactional logs and deprioritizing the replaying of the one or more transactional logs such that the replaying utilizes only unused resources of the first computing device.
2. The method of claim 1 further comprising storing the one or more DIFFs in a memory device of a second computing device.
3. The method of claim 1 wherein the database is a mailbox database for an email server.
4. The method of claim 1 wherein one or more of the DIFFs is either an independent DIFF or an embedded DIFF.
5. The method of claim 4 wherein one or more independent DIFFs are generated using a database file having a full database backup, a set of current backups of transactional logs, and a DIFF from a previous backup.
6. The method of claim 4 wherein one or more embedded DIFFs are generated using a database file, a set of current backups of transactional logs, and a set of DIFFs from all previous backups.
7. The method of claim 1 further comprising
receiving m transactional logs from the database;
creating one or more binary difference files of the database using the file tracker driver;
saving the one or more DIFFS created using the file tracker into the archive; and
saving the m transactional logs into the archive.
8. The method of claim 1 wherein one or more of the transactional logs are generated using a transactional system or a journaling system.
9. The method of claim 8 wherein the journaling system is New Technology File System.
10. The method of claim 8 wherein the transactional system is an email exchange server-based system.
11. The method of claim 1 further comprising generating a circular log comprising a plurality of the transactional logs.
12. A backup and restoration system for a transactional log based journaling application having a database comprising
a transactional log backup and restore process executing on one or more computing devices, wherein the transactional log backup and restore process selects a restore type that is initiated based on a selected point in time and using a first set of binary difference files (DIFF) for a first restore process or selecting a restore type that is initiated based on a fault or other point of failure and using a second set of DIFF for a second restore process;
an archive stored in non-transitory computer readable memory at a server;
a file tracker driver in electrical communication with and responsive to communication signals from the transactional backup process, wherein the file tracker driver is programmed to track changes in the database following a backup; and
a binary difference file generator at the server and in electronic communication with the archive and responsive to instructions from the transactional log backup process, wherein the binary difference file generator creates one DIFF of the database from one or more transactional logs of the database and the changes tracked by the file tracker driver, wherein the database runs on a first computing device, wherein the one or more DIFF comprise block by block differences relative to the database; wherein the transactional log backup and restore process is configured to reduce processing load on the first computing device by restoring one or more transactions to the database using the one or more DIFF instead of the transactional logs, wherein a plurality of DIFF is used to replay a virtualized database from prior backups and accumulated DIFF files to generate a restored database wherein the processing load of the first computing device is reduced by deprioritizing the replaying of the one or more transactional logs such that the replaying utilizes only unused resources of the first computing device.
13. The system of claim 12 wherein the binary difference file generator comprises an input component to receive the one or more transactional logs and output the one or more DIFFs.
14. The system of claim 13 wherein the one or more DIFFs is either an independent DIFF or an embedded DIFF.
15. The method of claim 1 further comprising applying one or more of the transactional logs to generate the one or more DIFFs.
16. A backup and restoration system comprising
a database comprising n transactional logs;
a file tracker driver, wherein the file tracker driver is programmed to track changes in the database following a backup;
a server comprising one or more processors, an archive running on the one or more processors, a binary difference file generator running on the one or more processors, wherein the binary difference file generator is configured to generate, using the file tracker driver, one or more binary difference files (DIFF) of the database from one or more of the n transactional log; and
a transactional log backup and restore process executing on one or more computing devices, wherein the transactional log backup and restore process selects a restore type that is initiated based on a selected point in time and using a first set of binary difference files (DIFF) for a first restore process or selecting a restore type that is initiated based on a fault or other point of failure and using a second set of DIFF for a second restore process;
wherein the transactional log backup and restore process is configured to reduce processing load on the one or more computing devices by restoring one or more transactions to the database using the one or more DIFF instead of the transactional logs wherein the processing load of the first computing device is reduced by deprioritizing the replaying of the one or more transactional logs such that the replaying utilizes only unused resources of one or more computing devices.