Defragmentation for objects within object store
Techniques are provided for managing objects within an object store. An object is maintained within an object store. The object comprises a plurality of slots. Each slot is used to store a unit of data accessible to applications hosted by remote computing devices. The object comprises an object header used to store metadata for each slot. A determination is made that the object is a fragmented object comprising an in-use slot of in-use data and a freed slot from which data was freed. The object is compacted to retain in-use data and exclude freed data as a rewritten object.
1. A method comprising:
maintaining an object, comprising a plurality of slots, within an object store, wherein each slot is used to store a unit of data accessible to applications hosted by remote computing devices, wherein the object comprises an object header used to store metadata for each slot;
creating a cloud block in-use map by walking a file system in a snapshot;
evaluating the cloud block in-use map to identify a freed slot of unused data no longer referenced by the remote computing devices;
determining that the object is a fragmented object comprising an in-use slot of in-use data and the freed slot; and
compacting the object to retain the in-use data and exclude the unused data as a rewritten object based upon a number of in-use slots being less than a threshold.
2. The method of claim 1 , comprising:
maintaining the metadata within the object header to specify locations of the in-use data within the object.
3. The method of claim 2 , wherein the in-use data is stored in new locations within the rewritten object, and wherein the method comprises:
updating the metadata with the new locations of the in-use data within the rewritten object.
4. The method of claim 1 , comprising:
copying snapshot data of a snapshot created by the remote computing device into a set of slots of the object.
5. The method of claim 4 , comprising:
determining that the set of slots are freed slots comprising unused data based upon the remote computing device deleting the snapshot.
6. The method of claim 5 , wherein the set of slots are determined to be freed slots based upon data within the slots being unique to the deleted snapshot.
7. The method of claim 4 , wherein the remote computing device deletes the snapshot, and the method comprising:
determining that the set of slots are in-use slots comprising in-use data based upon a second snapshot created by the remote computing device referencing data within the slots.
8. The method of claim 4 , wherein the remote computing device deletes the snapshot, and the method comprising:
determining that the set of slots are in-use slots comprising in-use data based upon data within the set of slots being non-unique to the deleted snapshot.
9. A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to:
maintain an object, comprising a plurality of slots, within an object store, wherein each slot is used to store a unit of data accessible to applications hosted by remote computing devices, wherein the object comprises an object header used to store metadata for each slot;
create a cloud block in-use map by walking a file system in a snapshot;
evaluate the cloud block in-use map to identify a freed slot of unused data no longer referenced by the remote computing devices;
determine that the object is a fragmented object comprising an in-use slot of in-use data and the freed slot; and
compact the object to retain the in-use data and exclude the unused data as a rewritten object based upon a number of in-use slots being less than a threshold.
10. The non-transitory machine readable medium of claim 9 , wherein the instructions cause the machine to:
maintain the metadata within the object header to specify locations of the in-use data within the object.
11. The non-transitory machine readable medium of claim 10 , wherein the in-use data is stored in new locations within the rewritten object, and wherein the instructions cause the machine to:
update the metadata with the new locations of the in-use data within the rewritten object.
12. The non-transitory machine readable medium of claim 9 , wherein the instructions cause the machine to:
copy snapshot data of a snapshot created by the remote computing device into a set of slots of the object.
13. The non-transitory machine readable medium of claim 12 , wherein the instructions cause the machine to:
determine that the set of slots are freed slots comprising unused data based upon the remote computing device deleting the snapshot.
14. The non-transitory machine readable medium of claim 13 , wherein the set of slots are determined to be freed slots based upon data within the slots being unique to the deleted snapshot.
15. The non-transitory machine readable medium of claim 12 , wherein the remote computing device deletes the snapshot, and wherein the instructions cause the machine to:
determine that the set of slots are in-use slots comprising in-use data based upon a second snapshot created by the remote computing device referencing data within the slots.
16. A computing device comprising:
a memory comprising machine executable code for performing a method; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
maintain an object, comprising a plurality of slots, within an object store, wherein each slot is used to store a unit of data accessible to applications hosted by remote computing devices, wherein the object comprises an object header used to store metadata for each slot;
create a cloud block in-use map by walking a file system in a snapshot;
evaluate the cloud block in-use map to identify a freed slot of unused data no longer referenced by the remote computing devices;
determine that the object is a fragmented object comprising an in-use slot of in-use data and the freed slot; and
compact the object to retain the in-use data and exclude the unused data as a rewritten object based upon a number of in-use slots being less than a threshold.
17. The computing device of claim 16 , wherein the machine executable code causes the processor to:
maintain the metadata within the object header to specify locations of the in-use data within the object.
18. The computing device of claim 17 , wherein the in-use data is stored in new locations within the rewritten object, and wherein the machine executable code causes the processor to:
update the metadata with the new locations of the in-use data within the rewritten object.
19. The computing device of claim 16 , wherein the machine executable code causes the processor to:
copy snapshot data of a snapshot created by the remote computing device into a set of slots of the object.
20. The computing device of claim 19 , wherein the cloud block in-use map comprises a bitmap.