Method and system for garbage collection in a storage system based on longevity of stored data
A method for managing data. The method includes receiving a first request to write data to persistent storage and in response to the first request, writing the data to a short-lived block in the persistent storage, where the data is short-lived data or data of unknown longevity. The method further includes performing a modified garbage collection operation that includes: selecting a first frag page in a first block, determining that the first frag page is live, and migrating, based on the determination that the first frag page is live, the first frag page to a long-lived block in the persistent storage, where the long-lived block is distinct from the short-lived block and wherein the long-lived block does not include any short-lived data.
1. A non-transitory computer readable medium comprising instructions, which when executed by a processor perform a method, the method comprising:
receiving a first request to write data to persistent storage;
in response to the first request and after determining that the data is non-persistent data, writing the data to a first block in the persistent storage; and
performing a modified garbage collection operation comprising:
selecting a frag page in a second block;
determining that the frag page comprises persistent data, wherein the first frag page comprises persistent data when the first frag page is live, and
migrating, based on the determination that the first frag page comprises persistent data, the first frag page to a third block in the persistent storage,
wherein the third block is distinct from the first block and wherein the third block does not include any non-persistent data, and
receiving a second request to write second data to the persistent storage, wherein the second request comprises information which indicates that the second data is persistent data; and
in response to the second request, writing the second data to the third block in the persistent storage.
2. The non-transitory computer readable medium of claim 1 , wherein determining that the first frag page comprises persistent data, comprises:
generating a first derived physical address using a first TOC entry associated with the first frag page;
selecting a first stored physical address based on a first object ID and a first offset ID, wherein the first object ID and the first offset ID are located in the first TOC entry; and
determining, based on the first derived physical address matching the first stored physical address, that the first frag page is live.
3. The non-transitory computer readable medium of claim 2 , wherein selecting the first stored physical address further comprises:
performing a lookup in an in-memory data structure using the first object ID and the first offset ID to identify the first stored physical address.
4. The non-transitory computer readable medium of claim 1 , wherein the first block is a NAND block.
5. The non-transitory computer readable medium of claim 1 , wherein the persistent storage is solid-state memory.
6. The non-transitory computer readable medium of claim 1 , wherein the modified garbage collection operation further comprises:
selecting a fourth block in the persistent storage;
determining the fourth block comprises a plurality of live frag pages and does not include any dead frag pages; and
migrating each live page of the plurality of live frag pages to a fifth block.
7. The non-transitory computer readable medium of claim 6 , wherein the migration of each of the plurality of live frag pages is performed for wear-leveling.
8. The non-transitory computer readable medium of claim 1 , wherein at least a portion of the modified garbage collection operation is performed in parallel with at least one selected from a group consisting of receiving the first request to write data to the persistent storage and writing the data to the first block in the persistent storage.
9. A method for managing data, comprising:
receiving a first request to write data to persistent storage;
in response to the first request and determining that the data is non-persistent data, writing the data to a first block in the persistent storage; and
performing a modified garbage collection operation comprising:
selecting a first frag page in a second block,
determining that the first frag page comprises persistent data, wherein the first frag page comprises persistent data when the first frag page is live, and
migrating, based on the determination that the first frag page comprises persistent data, the first frag page to a third block in the persistent storage,
wherein the third block is distinct from the first block and wherein the third block does not include any non-persistent data, and
receiving a second request to write second data to the persistent storage, wherein the second request comprises information which indicates that the second data is persistent data; and
in response to the second request, writing the second data to the third block in the persistent storage.
10. The method of claim 9 , wherein determining that the first frag page comprises persistent data, comprises:
generating a first derived physical address using a first TOC entry associated with the first frag page;
selecting a first stored physical address based on a first object ID and a first offset ID, wherein the first object ID and the first offset ID are located in the first TOC entry; and
determining, based on the first derived physical address matching the first stored physical address, the first frag page is live.
11. The method of claim 10 , wherein selecting the first stored physical address further comprises:
performing a lookup in an in-memory data structure using the first object ID and the first offset ID to identify the first stored physical address.
12. The method of claim 9 , wherein the modified garbage collection operation further comprises:
selecting a fourth block in the persistent storage;
determining the fourth block comprises a plurality of live frag pages and does not include any dead frag pages; and
migrating each live page of the plurality of live frag pages to a fifth block.
13. The method of claim 12 , wherein the migration of each of the plurality of live frag pages is performed for wear-leveling.
14. The method of claim 9 , wherein at least a portion of the modified garbage collection operation is performed in parallel with at least one selected from a group consisting of receiving the first request to write data to the persistent storage and writing the data to the first block in the persistent storage.
15. A storage appliance, comprising:
persistent storage;
a non-transitory computer readable medium comprising instructions;
a processor, configured to execute the instructions, wherein the instructions when executed by the processor perform a method, the method comprising:
receiving a first request to write data to the persistent storage;
in response to the first request and after determining that the data is non-persistent data, writing the data to a first block in the persistent storage; and
performing a modified garbage collection operation comprising:
selecting a first frag page in a second block
determining that the first frag page comprises persistent data, wherein the first frag page comprises persistent data when the first frag page is live, and
migrating, based on the determination that the first frag page comprises persistent data, the first frag page to a third block in the persistent storage,
wherein the third block is distinct from the first block and wherein the third block does not include any non-persistent data, and
receiving a second request to write second data to the persistent storage, wherein the second request comprises information which indicates that the second data is persistent data; and
in response to the second request, writing the second data to the third block in the persistent storage
wherein at least a portion of the modified garbage collection operation is performed in parallel with at least one selected from a group consisting of receiving the first request to write data to the persistent storage and writing the data to the first block in the persistent storage.
16. The storage appliance of claim 15 , wherein the persistent storage comprises a plurality of solid state memory modules.
17. The storage appliance of claim 15 , wherein determining that the first frag page comprises persistent data, comprises:
generating a first derived physical address using a first TOC entry associated with the first frag page;
selecting a first stored physical address based on a first object ID and a first offset ID, wherein the first object ID and the first offset ID are located in the first TOC entry; and
determining, based on the first derived physical address matching the first stored physical address, that the first frag page is live.
18. The storage appliance of claim 17 , further comprising:
memory configured to store an in-memory data structure,
wherein selecting the first stored physical address further comprises performing a lookup in the in-memory data structure using the first object ID and the first offset ID to identify the first stored physical address.