Method and apparatus for efficiently creating backup files with less redundancy
View Patent ↗A method of creating backup files having less redundancy. The method creates a backup file by creating an overhead segment for each file that is to be backed up and creating a data segment containing the data that is to be backed up for each file. After creating the overhead segment and the data segment, the overhead segment is placed into an overhead stream data segment is stored in memory. The overhead segment is also positioned in the overhead stream with a pointer that identifies the data segment within the memory. For backups of subsequent servers or the same server at a later time, the backup software will create a separate overhead stream. However, a plurality of overhead streams may contain pointers to the same data segments such that redundant data segments do not need to be stored in a backup server.
1. A method of creating backup files having less redundancy, comprising:
triggering a backup process;
for each file to be backed up, creating a data segment;
storing the data segment at a location in memory;
creating an overhead stream having a pointer that identifies the location of the data segment;
determining if the data segment is redundant to a previously stored data segment;
if the data segment is redundant, deleting the redundant data segment; and
updating the overhead stream pointer to identify the previously stored data segment that is identical to the deleted data segment.
2. The method of claim 1 wherein the determining step comprises:
computing a checksum that represents the contents of the data segment;
comparing the checksum to at least one previously generated checksums; and
if a matching checksum is found, identifying the data segment as redundant.
3. The method of claim 2 wherein the checksum is an MD5 checksum.
4. The method of claim 2 wherein the checksum is used to define a directory path and file name for the data segment.
5. The method of claim 2 further comprising, for each redundant data segment, performing a binary compare between the data segment and the previously stored data segment.
6. The method of claim 2 wherein the checksum is a unique signature or fingerprint representing the contents of the data segment.
7. The method of claim 1 wherein, each execution of the triggering step, creates a separate overhead stream, where a plurality of overhead streams identifies common data segments.
8. The method of claim 7 wherein each overhead stream represents a backup of a different device.
9. Apparatus for creating backup files having less redundancy comprising:
a backup server for accessing at least one file to be backed up, creating data segments and an overhead stream each time a backup is triggered, determining when a data segment is redundant; and when a data segment is redundant, deleting the data segment from memory and updating the overhead stream to identify a previously stored data segment that is identical to the deleted data segment.
10. The apparatus of claim 9 further comprising at least one server containing at least one file to be backed up.
11. The apparatus of claim 9 wherein a separate overhead stream is created for each at least one server.
12. A computer readable medium for storing software that, when executed on a processor, causes the processor to perform a method comprising:
for each file to be backed up, creating a data segment;
storing the data segment at a location in memory;
creating an overhead stream having a pointer that identifies the location of the data segment;
determining if the data segment is redundant to a previously stored data segment;
if the data segment is redundant, deleting the redundant data segment; and
updating the overhead stream pointer to identify the previously stored data segment that is identical to the deleted data segment.
13. The method of claim 12 wherein the determining step comprises:
computing a checksum that represents the contents of the data segment;
comparing the checksum to at least one previously generated checksums; and
if a matching checksum is found, identifying the data segment as redundant.
14. The method of claim 13 wherein the checksum is an MD5 checksum.
15. The method of claim 13 wherein the checksum is used to define a directory path and file name for the data segment.
16. The method of claim 15 further comprising, for each redundant data segment, performing a binary compare between the data segment and the previously stored data segment.
17. The method of claim 16 wherein the checksum is a unique signature or fingerprint representing the contents of the data segment.