Tiered data storage in flash memory based on write activity
A method includes receiving data objects for storage in at least one or more first storage devices characterized by a first write endurance, and one or more second storage devices characterized by a second write endurance that is lower than the first write endurance. Respective write-activity levels are estimated for the data objects. For each data object, a choice is made whether to store the data object in the first storage devices or in the second storage devices based on an estimated write-activity level of the data object. The data object is stored in the chosen storage devices.
1. A method, comprising:
receiving data objects for storage in at least:
(i) one or more first storage devices characterized by a first write endurance; and
(ii) one or more second storage devices characterized by a second write endurance that is lower than the first write endurance;
estimating respective write-activity levels for the data objects; and
for each data object:
(i) choosing whether to store the data object in the first storage devices or in the second storage devices based on an estimated write-activity level of the data object, and storing the data object in the chosen storage devices; and
(ii) storing metadata that is indicative of a storage location of the data object in the first or second storage devices, including, when the data object is stored in the second storage devices, storing authoritative metadata, comprising an authoritative indication that points to the storage location of the data object, and further storing cached-location metadata, comprising a first indication that aims to point to the storage location of the data object, and a second indication that points to the authoritative indication.
2. The method according to claim 1 , wherein the first storage devices comprise Multi-Level Cell (MLC) Flash memory, and wherein the second storage devices comprise Triple-Level Cell (TLC) Flash memory.
3. The method according to claim 1 , wherein the first storage devices and the second storage devices comprise respective different classes of Flash memory.
4. The method according to claim 1 , wherein choosing whether to store the data object in the first or second storage devices comprises choosing the first storage devices when the write-activity level exceeds a threshold, and choosing the second storage devices when the write-activity level is below the threshold.
5. The method according to claim 1 , wherein storing the data object comprises initially storing the data object in the first storage devices, and then optionally rewriting the data object to the second storage devices depending on the write-activity level.
6. The method according to claim 1 , and comprising accessing the data object using either the first indication or the second indication.
7. The method according to claim 1 , and comprising updating only the authoritative metadata and not the cached-location metadata upon a change in the storage location of the data object.
8. The method according to claim 1 , and comprising aggregating a plurality of deletions of data objects from the second storage devices, and updating the metadata jointly with the aggregated deletions.
9. The method according to claim 1 , and comprising accumulating a plurality of the data objects that are destined for storage in the second storage devices, and then storing the accumulated plurality.
10. The method according to claim 9 , wherein storing the accumulated plurality comprises performing at least one of compressing the plurality of the data objects and applying a redundant storage scheme to the accumulated plurality of the data objects.
11. A system, comprising:
one or more first storage devices characterized by a first write endurance;
one or more second storage devices characterized by a second write endurance that is lower than the first write endurance; and
one or more processors, which are configured to receive data objects for storage, to estimate respective write-activity levels for the data objects, and, for each data object:
(i) to choose whether to store the data object in the first storage devices or in the second storage devices based on an estimated write-activity level of the data object, and to store the data object in the chosen storage devices; and
(ii) to store metadata that is indicative of a storage location of the data object in the first or second storage devices, including, when the data object is stored in the second storage devices, to store authoritative metadata, comprising an authoritative indication that points to the storage location of the data object, and to further store cached-location metadata, comprising a first indication that aims to point to the storage location of the data object, and a second indication that points to the authoritative indication.
12. The system according to claim 11 , wherein the first storage devices comprise Multi-Level Cell (MLC) Flash memory, and wherein the second storage devices comprise Triple-Level Cell (TLC) Flash memory.
13. The system according to claim 11 , wherein the first storage devices and the second storage devices comprise respective different classes of Flash memory.
14. The system according to claim 11 , wherein the processors are configured to choose to store the data object in the first storage devices when the write-activity level exceeds a threshold, and to choose to store the data object in the second storage devices when the write-activity level is below the threshold.
15. The system according to claim 11 , wherein the processors are configured to initially store the data object in the first storage devices, and then optionally rewrite the data object to the second storage devices depending on the write-activity level.
16. The system according to claim 11 , wherein the processors are configured to access the data object using either the first indication or the second indication.
17. The system according to claim 11 , wherein the processors are configured to update only the authoritative metadata and not the cached-location metadata upon a change in the storage location of the data object.
18. The system according to claim 11 , wherein the processors are configured to aggregate a plurality of deletions of data objects from the second storage devices, and to update the metadata jointly with the aggregated deletions.
19. The system according to claim 11 , wherein the processors are configured to accumulate a plurality of the data objects that are destined for storage in the second storage devices, and then to store the accumulated plurality.
20. The system according to claim 19 , wherein the processors are configured to perform at least one of compressing the plurality of the data objects and applying a redundant storage scheme to the accumulated plurality of the data objects.