Method and system for implementing fast incremental backups
A method for fast incremental replication of a file system. The method includes, at a data storage level, tracking file system status for a plurality of files by using a data structure inside respective inodes for the plurality of files, and at a file system level, tracking file system status for a plurality of files by using a file system mask structure. For all files present in a backup, a catalogue of the file path name and inode number is maintained on a source file system. During incremental backup, a source file system data structure enumerating a plurality of modified inodes is consulted. For inodes that have not been marked as removed or created, the file system mask structure that tracks modified portions is consulted to obtain a file incremental change and inodes. The method further includes performing incremental backup using the inodes that have been modified and appended.
1. A method for fast incremental replication of a file system, comprising:
at a data storage level, tracking a file system status for a plurality of files by using a data structure inside respective inodes for the plurality of files;
at a file system level, tracking a file system status for a plurality of files by using a file system mask structure;
for all files present in a backup, maintaining a catalogue of a file path name and inode number on a source file system;
during incremental backup, consulting a source file system data structure enumerating a plurality of modified inodes to determine inodes that have been marked as removed or created;
for inodes that have not been marked as removed or created, consulting the file system mask structure that tracks modified portions to obtain a file incremental change and inodes that have been modified and appended, the file incremental change indicating a difference between a stored file size and a subsequent file size; and
performing incremental backup using the inodes that have been modified and appended.
2. The method of claim 1 , wherein when a file has been removed, an on-disk inode tracks the same separately by using a bit field.
3. The method of claim 1 , wherein when a file has been created, an on-disk inode tracks the same separately by using a bit field, and regarding said file creation, no file range modification is done.
4. The method of claim 1 , wherein each file in the file system tracks ranges modified by maintaining such information persistently by dividing said each file into chunks and representing the chunks by a bit-map.
5. The method of claim 1 , wherein each file in the file system tracks ranges modified by maintaining such information persistent on at the data storage level.
6. The method of claim 1 , wherein each time a file is created or renamed, said file's name and parent directory inode number is recorded with the inode at the data storage level.
7. The method of claim 1 , wherein once a directory inode that is not marked as created has been reached, a path is extracted from an inode number using a catalog maintained at a backup.
8. A computer readable storage medium having stored thereon computer executable instructions that, when executed by a computer system, cause the computer system to perform a method comprising:
at a data storage level, tracking a file system status for a plurality of files by using a data structure inside respective inodes for the plurality of files;
at a file system level, tracking a file system status for a plurality of files by using a file system mask structure;
for all files present in a backup, maintaining a catalogue of the file path name and inode number on the source file system;
during incremental backup, consulting a source file system data structure enumerating a plurality of modified inodes to determine inodes that have been marked as removed or created;
for inodes that have not been marked as removed or created, consulting the file system mask structure that tracks modified portions to obtain a file incremental change and inodes that have been modified and appended, the file incremental change indicating a difference between a stored file size and a subsequent file size; and
performing incremental backup using the inodes that have been modified and appended.
9. The computer readable storage medium of claim 8 , wherein when a file has been removed, an on-disk inode tracks the same separately by using a bit field.
10. The computer readable storage medium of claim 8 , wherein when a file has been created, an on-disk inode tracks the same separately by using a bit field.
11. The computer readable storage medium of claim 8 , wherein each file in the file system tracks ranges modified by maintaining such information persistently by dividing said each file into chunks and representing the chunks by a bit-map.
12. The computer readable storage medium of claim 8 , wherein each file in the file system tracks ranges modified by maintaining such information persistent on at the data storage level.
13. The computer readable storage medium of claim 8 , wherein each time a file is created or renamed, said file's name and parent directory inode number is recorded with the inode at the data storage level.
14. The computer readable storage medium of claim 8 , wherein once a directory inode that is not marked as created has been reached, a path is extracted from an inode number using a catalog maintained at a backup.
15. A computer system having a microprocessor coupled to a computer readable memory, the memory having computer readable code therein which when executed by the computer system causes the computer system to implement a method comprising:
at a data storage level, tracking a file system status for a plurality of files by using a data structure inside respective inodes for the plurality of files;
at a file system level, tracking a file system status for a plurality of files by using a file system mask structure;
for all files present in a backup, maintaining a catalogue of the file path name and inode number on the source file system;
during incremental backup, consulting a source file system data structure enumerating a plurality of modified inodes to determine inodes that have been marked as removed or created;
for inodes that have not been marked as removed or created, consulting the file system mask structure that tracks modified portions to obtain a file incremental change and inodes that have been modified and appended, the file incremental change indicating a difference between a stored file size and a subsequent file size;
performing incremental backup using the inodes that have been modified and appended.
16. The computer system of claim 15 , wherein when a file has been removed, an on-disk inode tracks the same separately by using a bit field.
17. The computer system of claim 15 , wherein when a file has been created, an on-disk inode tracks the same separately by using a bit field, and regarding said file creation, no file range modification is done.
18. The computer system of claim 15 , wherein each file in the file system tracks ranges modified by maintaining such information persistently by dividing said each file into chunks and representing the chunks by a bit-map.
19. The computer system of claim 15 , wherein each file in the file system tracks ranges modified by maintaining such information persistent on at the data storage level.
20. The computer system of claim 15 , wherein each time a file is created or renamed, said file's name and parent directory inode number is recorded with the inode at the data storage level.