Elastic metadata and multiple tray allocation
In some examples, both object data and corresponding metadata are stored on thin provisioned block storage in which blocks for the storage of the object data and the metadata are dynamically provisioned from a shared pool. For instance, the metadata may be stored on the block storage using a tree data structure that may grow proportionally with the corresponding stored object data. The object storage may automatically adapt to the size and number of objects being stored, and storage space for metadata may be allocated dynamically on demand. Additionally, some implementations include storing data across multiple trays. For instance, a plurality of storage extents of an extent group may be allocated across a plurality of storage devices on multiple trays. The extent allocation may be tray-aware by allocating extents to maximize the probability that data remains available/recoverable in the presence of a failure in one or more trays.
1. A system comprising:
a plurality of trays, each tray including a plurality of storage devices, each storage device having multiple storage blocks for storing data, wherein a plurality of storage extents are configurable on the storage devices, each storage extent configured from a plurality of the storage blocks, each tray and respective storage devices being distinguishable from other trays and respective storage devices; and
one or more processors coupled to the plurality of trays, the processors programmed by executable instructions to:
determine a number of data extents and a number of coding extents to allocate for creating a first extent group and a second extent group;
allocate the data extents and the coding extents on the storage devices on the plurality of trays so that at least a first tray of the plurality of trays has a total number of allocated extents for the first extent group that is less than or equal to the number of coding extents;
store object data on the first extent group; and
store metadata corresponding to the object data as a tree data structure directly in the storage blocks of the second extent group.
2. The system as recited in claim 1 , wherein the one or more processors are further programmed to allocate the extents of the first extent group according to a first protection class such that the first extent group includes a first number of data extents and a second number of coding extents, wherein the coding extents enable, at least in part, data recovery for at least one of the data extents in the event that a storage device on which the at least one data extent is allocated becomes unavailable.
3. The system as recited in claim 2 , wherein the one or more processors are further programmed to allocate the extents of the second extent group on the plurality of trays so that at least a second tray of the plurality of trays has a total number of allocated extents for the second extent group that is less than or equal to the number of coding extents, wherein the second extent group is allocated according to a second protection class such that the second extent group includes at least one of:
a different number of data extents from the first extent group; or
a different number of coding extents from the first extent group.
4. The system as recited in claim 1 , wherein the one or more processors are further programmed to:
receive, from a client, a write request;
request allocation of one or more additional extents to the second extent group for storing metadata corresponding to the write request in the tree data structure; and
store metadata corresponding to the write request in the tree data structure on the one or more additional extents allocated to the second extent group.
5. The system as recited in claim 4 , wherein the one or more processors are further programmed to:
request allocation of one or more additional extents to the first extent group for storing data corresponding to the write request; and
store at least a portion of the data corresponding to the write request on the one or more additional extents allocated to the first extent group.
6. The system as recited in claim 4 , wherein the one or more processors are further programmed to:
based on storage of the data corresponding to the write request, add file metadata about the data to a table maintaining file information for files stored on the first extent group; and
update the tree data structure with the file metadata added to the table.
7. The system as recited in claim 1 , wherein the one or more processors are further programmed to allocate the data extents and the coding extents to the plurality of trays such that, following failure of one of the trays, a number of allocated extents remaining available on other trays of the plurality of trays is greater than or equal to the number of data extents,
wherein at least one of the coding extents enables, at least in part, data recovery for at least one data extent in the event that a storage device on which the at least one data extent is allocated becomes unavailable.
8. The system as recited in claim 1 , wherein the one or more processors are further programmed to determine a cutoff threshold for the first extent group based at least in part on the number of data extents, the number of coding extents, and a number of trays having storage devices with available extents, wherein, a number of allocated extents on a second tray of the plurality of trays corresponds to the cutoff threshold.
9. The system as recited in claim 8 , wherein the one or more processors are further programmed to:
determine a total number extents to be allocated based on the number of data extents plus the number of coding extents for the first extent group;
determine a product of the number of trays, minus one, times the number of coding extents; and
determine the cutoff threshold from a difference between the total extents to be allocated and the product.
10. The system as recited in claim 8 , wherein the one or more processors are further programmed to:
add another tray to the plurality of trays;
move at least one allocated extent from the second tray to the added other tray;
recalculate the cutoff threshold based on adding the other tray to the plurality of trays; and
determine a number of allocated extents to move from the second tray to the added other tray based at least in part on the recalculated cutoff threshold.
11. The system as recited in claim 1 , wherein the one or more processors are further programmed to allocate the extents for the first extent group across the plurality of storage devices on the plurality of trays, respectively, so that no more than one extent for the first extent group is allocated on each storage device.
12. A method executed by one or more processors of a node in communication with a plurality of trays, each tray including a plurality of storage devices, each storage device having multiple storage blocks for storing data, wherein a plurality of storage extents are configurable on the storage devices, each storage extent configured from a plurality of the storage blocks, each tray and respective storage devices being distinguishable from other trays and respective storage devices, the method comprising:
determining, by the one or more processors, a number of data extents and a number of coding extents to allocate for creating a first extent group and a second extent group;
allocating, by the one or more processors, the data extents and the coding extents on the storage devices on the plurality of trays so that at least a first tray of the plurality of trays has a total number of allocated extents for the first extent group that is less than or equal to the number of coding extents;
storing, by the one or more processors, object data on the first extent group; and
storing, by the one or more processors, metadata corresponding to the object data as a tree data structure directly in the storage blocks of the second extent group.
13. The method as recited in claim 12 , further comprising:
allocating the data extents and the coding extents to the plurality of trays such that, following failure of one of the trays, a number of allocated extents remaining available on other trays of the plurality of trays is greater than or equal to the number of data extents,
wherein at least one of the coding extents enables, at least in part, data recovery for at least one data extent in the event that a storage device on which the at least one data extent is allocated becomes unavailable.
14. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, program the one or more processors to:
execute a block storage service for accessing a plurality of trays, each tray including a plurality of storage devices, each storage device having multiple storage blocks for storing data, wherein a plurality of storage extents are configurable on the storage devices, each storage extent configured from a plurality of the storage blocks, each tray and respective storage devices being distinguishable from other trays and respective storage devices;
determine a number of data extents and a number of coding extents to allocate for creating a first extent group and a second extent group;
allocate the data extents and the coding extents on the storage devices on the plurality of trays so that at least a first tray of the plurality of trays has a total number of allocated extents for the first extent group that is less than or equal to the number of coding extents;
store object data on the first extent group; and
store metadata corresponding to the object data as a tree data structure directly in the storage blocks of the second extent group.
15. The one or more non-transitory computer-readable media as recited in claim 14 , wherein the instructions further program the one or more processors to:
allocate the extents of the first extent group according to a first protection class such that the first extent group includes a first number of data extents and a second number of coding extents, wherein the coding extents enable, at least in part, data recovery for at least one of the data extents in the event that a storage device on which the at least one data extent is allocated becomes unavailable; and
allocate the extents of the second extent group on the plurality of trays so that at least a second tray of the plurality of trays has a total number of allocated extents for the second extent group that is less than or equal to the number of coding extents, wherein the second extent group is allocated according to a second protection class such that the second extent group includes at least one of:
a different number of data extents from the first extent group; or
a different number of coding extents from the first extent group.
16. The one or more non-transitory computer-readable media as recited in claim 14 , wherein the instructions further program the one or more processors to allocate the data extents and the coding extents to the plurality of trays such that, following failure of one of the trays, a number of allocated extents remaining available on other trays of the plurality of trays is greater than or equal to the number of data extents,
wherein at least one of the coding extents enables, at least in part, data recovery for at least one data extent in the event that a storage device on which the at least one data extent is allocated becomes unavailable.
17. The one or more non-transitory computer-readable media as recited in claim 14 , wherein the instructions further program the one or more processors to determine a cutoff threshold for the first extent group based at least in part on the number of data extents, the number of coding extents, and a number of trays having storage devices with available extents, wherein, a number of allocated extents on a second tray of the plurality of trays corresponds to the cutoff threshold.
18. The one or more non-transitory computer-readable media as recited in claim 17 , wherein the instructions further program the one or more processors to:
determine a total number extents to be allocated based on the number of data extents plus the number of coding extents for the first extent group;
determine a product of the number of trays, minus one, times the number of coding extents; and
determine the cutoff threshold from a difference between the total extents to be allocated and the product.
19. The one or more non-transitory computer-readable media as recited in claim 14 , wherein the instructions further program the one or more processors to:
add another tray to the plurality of trays;
move at least one allocated extent from the second tray to the added other tray;
recalculate the cutoff threshold based on adding the other tray to the plurality of trays; and
determine a number of allocated extents to move from the second tray to the added other tray based at least in part on the recalculated cutoff threshold.
20. The one or more non-transitory computer-readable media as recited in claim 14 , wherein the instructions further program the one or more processors to allocate the extents for the first extent group across the plurality of storage devices on the plurality of trays, respectively, so that no more than one extent for the first extent group is allocated on each storage device.