Summary file change log for faster forever incremental backup
Systems and methods for performing backups including incremental backups are provided. A change log is provided. The change log includes a map file that includes a bit for each inode in a file system. The bits are set on certain file events. The map file allows the files to be included in the next backup to be identified without having to trawl the entire file system. The change log also includes a data file that lists actual changes to the various files in corresponding records that are indexed by the inode number. The change log can enable faster incremental backups in part by eliminating unnecessary disk access requests.
1. A method for performing a backup of a file system in a computing system, the method comprising:
maintaining a change log for the file system, wherein the change log includes events that occur to files in the file system that have changed since a most recent backup, wherein the change log includes a bitmap and a data file, wherein the bitmap includes an entry for each file in the file system and the data file includes a record of entries for each file, each entry corresponding to an event;
detecting the events in the file system and processing the events to determine whether files associated with the events are modified, wherein at least some of the events that modify the files do not result in an entry in the change log, and wherein at least some of the events result in an entry in the bitmap file but not in the data file;
identifying files in the file system to include in the backup operation by accessing the change log;
for every file in the change log that has changed since the most recent backup, determining whether to perform the backup operation for the file based on the associated events of the file in the change log; and
performing the backup operation on the file system by backing up only the files whose associated events indicate that backup is required, wherein disk accesses for files that do not require backup based on the associated events are avoided.
2. The method of claim 1 , further comprising creating the entries when the events are detected for the files.
3. The method of claim 1 , further comprising accessing the bitmap file to generate a list of files to be included in the backup operation, wherein the backup operation is an incremental backup operation.
4. The method of claim 3 , further comprising processing all events associated with a file before determining whether to perform the incremental backup operation.
5. The method of claim 4 , further comprising removing files from the list that do not need to be included in the incremental backup operation.
6. The method of claim 1 , wherein the file system includes inodes and each of the files is associated with one of the inodes, wherein the bitmap file and the data file are based on a number of inodes in the file system.
7. The method of claim 1 , wherein the record includes one or more of an old file name, a new file name, an event mask, a file size, and an inode number.
8. A non-transitory computer readable medium comprising computer executable instructions configured to be executed by a processor to implement a method for performing a backup of a file system in a computing system, the method comprising:
maintaining a change log for the file system, wherein the change log includes events that occur to files in the file system that have changed since a most recent backup, wherein the change log includes a bitmap and a data file, wherein the bitmap includes an entry for each file in the file system and the data file includes a record of entries for each file, each entry corresponding to an event;
detecting the events in the file system and processing the events to determine whether files associated with the events are modified, wherein at least some of the events that modify the files do not result in an entry in the change log, and wherein at least some of the events result in an entry in the bitmap file but not in the data file;
identifying files in the file system to include in the backup operation by accessing the change log;
for every file in the change log that has changed since the most recent backup, determining whether to perform the backup operation for the file based on the associated events of the file in the change log; and
performing the backup operation on the file system by backing up only the files whose associated events indicate that backup is required, wherein disk accesses for files that do not require backup based on the associated events are avoided.
9. The non-transitory computer readable medium of claim 8 , the method further comprising creating the entries when the events are detected for the files.
10. The non-transitory computer readable medium of claim 8 , the method further comprising accessing the bitmap file to generate a list of files to be included in the backup operation, wherein the backup operation is an incremental backup operation.
11. The non-transitory computer readable medium of claim 10 , the method further comprising processing all events associated with a file before determining whether to perform the incremental backup operation.
12. The non-transitory computer readable medium of claim 11 , the method further comprising removing files from the list that do not need to be included in the incremental backup operation.
13. The non-transitory computer readable medium of claim 8 , wherein the file system includes inodes and each of the files is associated with one of the inodes, wherein the bitmap file and the data file are based on a number of inodes in the file system.
14. The non-transitory computer readable medium of claim 8 , wherein the record includes one or more of an old file name, a new file name, an event mask, a file size, and an inode number.