Solid state drive cache recovery in a clustered storage system
A storage system that includes multiple nodes, each node comprises a SSD cache and a management module and hard disk drives that are coupled to the nodes. The management module of each node is arranged to manage a SSD cache map that comprises multiple entries for storing mappings from logical addresses to SSD cache physical addresses and to physical addresses in the hard disk drives. The mappings are related to data units stored in the SSD cache. Upon a rejoin of a certain node following a shut-down or a failure of the certain node, the certain node is arranged to: obtain from at least one other node, current mappings between logical addresses and physical addresses in the hard disk drives, and perform a validation process of the data units stored in the SSD cache in response to relationships between the current mappings and the entries of the SSD cache map.
1. A storage system that comprises:
multiple nodes, each node comprises a solid state drive (SSD) cache and a management module;
hard disk drives that are coupled to the nodes;
wherein the management module of each node is arranged to manage a SSD cache map that comprises multiple entries for storing mappings from logical addresses to SSD cache physical addresses and to physical addresses in the hard disk drives; wherein the mappings are related to data units stored in the SSD cache;
wherein upon a rejoin of a certain node following a shut-down or a failure of the certain node, the certain node is arranged to:
obtain from at least one other node, current mappings between logical addresses and hard disk drive (HDD) physical addresses, wherein the HDD physical addresses are indicative of addresses in the hard disk drives, and
perform a validation process of the data units stored in the SSD cache of the certain node in response to relationships between the current mappings and the entries of the SSD cache map.
2. The storage system according to claim 1 , wherein the certain node is arranged to invalidate an entry of the SSD cache map that stores a certain logical address if a mapping of that certain logical address to a physical address as reflected by the entry of the SSD cache map differs from a mapping of the certain logical address as reflected by the current mappings.
3. The storage system according to claim 1 , wherein the certain node is arranged to reject SSD cache read requests until the validation process is completed.
4. The storage system according to claim 1 , wherein the certain node is arranged to maintain a hard disk drives (HDDs) map that maps logical addresses to physical addresses; wherein upon the rejoin, the certain node is arranged to update the HDDs map to reflect the current mapping between logical addresses and physical addresses.
5. The storage system according to claim 1 , wherein each node is arranged to store in the SSD cache only data units that were already stored in a hard disk drive.
6. The storage system according to claim 1 , wherein the certain node is arranged to invalidate an entry of the SSD cache map that refers to a certain physical address and to a certain logical address if it is suspected that while the certain node was in a failed state, another node wrote to the certain physical address a data unit that was referred by the certain logical address.
7. The storage system according to claim 1 , wherein the certain node is arranged to invalidate an entry of the SSD cache map that refers to a certain physical address and to a certain logical address if it is suspected that while the certain node was in a failed state, another node wrote to the certain logical address a data unit regardless of a physical address allocated to the certain logical address.
8. The storage system according to claim 1 , wherein the SSD cache map entries store timestamps indicative of a time of writing data units to the SSD cache.
9. The storage system according to claim 8 , wherein the certain node is arranged to invalidate, during the validation process and in response to the timestamps, SSD cache map entries that store data units that were written to the SSD cache more than a predetermined period before the validation process.
10. The storage system according to claim 9 , wherein a length of the predetermined period is determined in accordance to an expected duration between two successful write operations to a same physical address.
11. The storage system according to claim 9 , wherein a length of the predetermined period is determined in accordance to an expected duration between two successful write operations of data units of a same logical address to a same physical address.
12. The storage system according to claim 1 , wherein the certain node is arranged to invalidate an entry of the SSD cache map that refers to a certain physical address and to a certain logical address if detecting an invalidation event that occurred while the certain node was in a failed state, and the invalidation event includes writing by another node to the certain physical address a data unit that was referred by the certain logical address.
13. The storage system according to claim 1 , wherein the certain node is arranged to invalidate an entry of the SSD cache map that refers to a certain physical address and to a certain logical address if detecting an invalidation event that occurred while the certain node was in a failed state, and the invalidation event included writing a data unit by another node to the certain physical address.
14. The storage system according to claim 13 , wherein the certain node is arranged to detect the invalidation event by processing physical address re-allocation metadata indicative of physical address re-allocation events.
15. The storage system according to claim 14 , wherein the SSD cache map further stores physical address re-allocation metadata and wherein the certain node is arranged to detect the invalidation event by comparing between physical address re-allocation metadata value and current value of physical address re-allocation metadata.
16. A method, comprising:
maintaining, by each node out of multiple nodes of a storage system, a solid state disk (SSD) cache map that comprises multiple entries for storing mappings from logical addresses to SSD cache physical addresses and to hard disk drives (HDDs) physical addresses; wherein the mappings are related to data units stored in the SSD cache; wherein the multiple nodes are coupled to hard disk drives and each node comprises a solid state drive (SSD) cache and a management module; and
performing a recovery process of a certain node, upon a rejoin of the certain node following a shut-down or a failure of the certain node, wherein the recovery process comprises:
obtaining from at least one other node, current mappings between logical addresses and HDDs physical addresses, wherein the HDDs physical addresses are indicative of addresses in the hard disk drives, and
performing a validation process of the data units stored in the SSD cache of the certain node in response to relationships between the current mappings and the entries of the SSD cache map.
17. The method according to claim 16 , comprising invalidating an entry of the SSD cache map that stores a certain logical address if a mapping of that certain logical address to a physical address as reflected by the entry of the SSD cache map differs from a mapping of the certain logical address as reflected by the current mappings.
18. The method according to claim 16 , comprising rejecting SSD cache read requests until the validation process is completed.
19. The method according to claim 16 , comprising detecting an invalidation event by processing physical address re-allocation metadata indicative of physical address re-allocation events.
20. A non-transitory computer readable medium that stores instructions for:
maintaining, by each node out of multiple nodes of a storage system, a solid state disk (SSD) cache map that comprises multiple entries for storing mappings from logical addresses to SSD cache physical addresses and to hard disk drives (HDDs) physical addresses; wherein the multiple nodes are coupled to hard disk drives and each node comprises a SSD cache and a management module; and
performing a recovery process of a certain node upon a rejoin of a certain node following shut-down or a failure of the certain node, wherein the recovery process comprises (a) obtaining from at least one other node, current mappings between logical addresses and HDDs physical addresses, wherein the HDDs physical addresses are indicative of addresses in the hard disk drives, and (b) performing a validation process of the entries of the SSD cache of the certain node is response to relationships between the current mappings and the entries of the SSD cache map.