Transferring snapshot copy to object store with deduplication preservation and additional compression
Techniques are provided for incremental snapshot copy to an object store. A list of deallocated block numbers of primary storage of a computing device are identified. Entries for the list of deallocated block numbers are removed from a mapping metafile. A list of changed block numbers corresponding to changes between a current snapshot of the primary storage and a prior copied snapshot copied from the primary storage to the object store is determined. The mapping metafile is evaluated using the list of changed block numbers to identify a deduplicated set of changed block numbers without entries within the mapping metafile. An object, comprising data of the deduplicated set of changed block numbers, is transmitted to the object store for storage as a new copied snapshot.
1. A method comprising:
generating an object comprising data of compressed block numbers;
generating an indicator to indicate that the object comprises the data of the compressed block numbers;
storing the object within an object store; and
populating a mapping metafile with an entry for a block number of data stored within the object, wherein the entry maps the block number to a cloud block number associated with the object store, and wherein the entry is populated with a compression group start indicator to indicate whether the block number is a starting block number for a compression group of a plurality of block numbers of compressed data blocks in the compression group, and wherein the mapping metafile is maintained separate from the object.
2. The method of claim 1 , wherein the compressed block numbers correspond to data blocks within primary storage, and the method comprising:
determining a list of deallocated block numbers of the primary storage; and
removing entries for the list of deallocated block numbers from the mapping metafile.
3. The method of claim 1 , comprising:
populating the entry with a compression indicator to indicate whether data of the block number is compressed.
4. The method of claim 1 , wherein the compressed block numbers correspond to data blocks within primary storage, and the method comprising:
determining a list of changed block numbers corresponding to changes between a current snapshot of the primary storage and a prior copied snapshot copied from the primary storage to the object store.
5. The method of claim 1 , comprising:
populating the entry with an overflow indicator to indicate whether the data block has an overflow entry within an overflow mapping metafile.
6. The method of claim 1 , comprising:
populating the entry with a logical length of an extent associated with the block number.
7. The method of claim 1 , comprising:
populating the entry with a physical length of an extent associated with the block number.
8. The method of claim 1 , wherein the compressed block numbers correspond to data blocks within primary storage that is hosted remote to the object store that is by a cloud computing environment.
9. The method of claim 1 , wherein the compressed block numbers correspond to data blocks within primary storage, and the method comprising:
indexing the mapping metafile by the compressed block numbers of the primary storage, wherein a compressed block number corresponds to a virtual volume block number.
10. A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to:
generate an object comprising data of compressed block numbers of data stored within the object;
generate an indicator to indicate that the object comprises the data of the compressed block numbers;
store the object within an object store; and
populate a mapping metafile with an entry for a block number of data stored within the object, wherein the entry maps the block number to a cloud block number associated with the object store, and wherein the entry is populated with a compression group start indicator to indicate whether the block number is a starting block number for a compression group of a plurality of block numbers of compressed data blocks in the compression group, and wherein the mapping metafile is maintained separate from the object.
11. The non-transitory machine readable medium of claim 10 , wherein the compressed block numbers correspond to data blocks within primary storage, and wherein the instructions cause the machine to:
determine a list of deallocated block numbers of the primary storage; and
remove entries for the list of deallocated block numbers from the mapping metafile.
12. The non-transitory machine readable medium of claim 10 , wherein the instructions cause the machine to:
populate the entry with a compression indicator to indicate whether data of the block number is compressed.
13. The non-transitory machine readable medium of claim 10 , wherein the compressed block numbers correspond to data blocks within primary storage, and wherein the instructions cause the machine to:
determine a list of changed block numbers corresponding to changes between a current snapshot of the primary storage and a prior copied snapshot copied from the primary storage to the object store.
14. The non-transitory machine readable medium of claim 10 , wherein the instructions cause the machine to:
populate the entry with an overflow indicator to indicate whether the data block has an overflow entry within an overflow mapping metafile.
15. The non-transitory machine readable medium of claim 10 , wherein the instructions cause the machine to:
populate the entry with a logical length of an extent associated with the block number.
16. The non-transitory machine readable medium of claim 10 , wherein the instructions cause the machine to:
populate the entry with a physical length of an extent associated with the block number.
17. The non-transitory machine readable medium of claim 10 , wherein the compressed block numbers correspond to data blocks within primary storage that is hosted remote to the object store that is by a cloud computing environment.
18. The non-transitory machine readable medium of claim 10 , wherein the compressed block numbers correspond to data blocks within primary storage, and wherein the instructions cause the machine to:
index the mapping metafile by the compressed block numbers of the primary storage, wherein a compressed block number corresponds to a virtual volume block number.
19. A computing device, comprising:
a memory comprising machine executable code; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
generate an object comprising data of compressed block numbers of data stored within the object;
generate an indicator to indicate that the object comprises the data of the compressed block numbers;
store the object within an object store; and
populate a mapping metafile with an entry for a block number of data stored within the object, wherein the entry maps the block number to a cloud block number associated with the object store, and wherein the entry is populated with a compression group start indicator to indicate whether the block number is a starting block number for a compression group of a plurality of block numbers of compressed data blocks in the compression group, and wherein the mapping metafile is maintained separate from the object.
20. The computing device of claim 19 , wherein the compressed block numbers correspond to data blocks within primary storage, and wherein the machine executable code causes the processor to:
determine a list of deallocated block numbers of the primary storage; and
remove entries for the list of deallocated block numbers from the mapping metafile.