Automated management of write streams for multi-tenant storage
Techniques for processing I/O operations may include: randomly assigning extent identifiers of extents to RAID groups; receiving write activity information for the extents for a sampling period; for each of the RAID groups, performing stream detection processing in accordance with the first write activity information and determining stream groups, wherein each stream group of one of the RAID groups includes extents of the RAID group belonging to a same write stream in the RAID group, wherein stream groups across all RAID groups are uniquely identified using stream group identifiers; tagging a write operation directed to one of the extents with a first stream group identifier denoting a stream group including the one extent; and issuing the write operation to a solid state drive of one of the RAID groups. Stream detection may use K-Means clustering algorithm. An aggregate address range of logical devices is mapped to the extent identifiers.
1. A method of processing I/O operations comprising:
randomly assigning a plurality of extent identifiers to a plurality of RAID (redundant array of independent disks) groups, wherein each of the plurality of extent identifiers denotes a different one of a plurality of extents;
receiving first write activity information for the plurality of extents for a first sampling period;
for each of the plurality of RAID groups, performing stream detection processing in accordance with the first write activity information and determining a plurality of stream groups, wherein each of the plurality of stream groups for each of the plurality of RAID groups includes extents of said each RAID group belonging to a same write stream in said each RAID group, wherein each of the plurality of stream groups of each of the plurality of RAID groups is uniquely identified using one of a plurality of stream group identifiers;
tagging a write operation directed to one of the plurality of extents with a first of the plurality of stream group identifiers denoting a stream group including the one extent; and
issuing the write operation to a solid state drive of one of the plurality of RAID groups.
2. The method of claim 1 , wherein said stream detection processing for a first of the plurality of RAID groups includes:
determining, using a clustering algorithm and in accordance with the first write activity information, a plurality of clusters of extents of the first RAID group, wherein each of the plurality of clusters of extents of the first RAID group identifies a different write stream of extents of the first RAID group.
3. The method of claim 2 , wherein the clustering algorithm is a k-means clustering algorithm.
4. The method of claim 2 , wherein each of the plurality of clusters of extents of the first RAID group includes extents of the first RAID group determined by the clustering algorithm as similar with respect to a plurality of dimensions.
5. The method of claim 4 , wherein the plurality of dimensions include extent identifier and write activity.
6. The method of claim 5 , wherein each of the plurality of clusters of extents of the first RAID group includes extents of the first RAID group determined by the clustering algorithm to have similar write activity and also similar locality of reference based on extent identifiers associated with extents of said each cluster.
7. The method of claim 6 , wherein the first write activity information for each of the plurality of extents includes an amount of data written to said each extent per unit of time during the first sampling period.
8. The method of claim 1 , further comprising:
forming an aggregate LBA (logical block address range) across contiguous LBA ranges of a plurality of logical devices; and
sequentially mapping portions of the aggregate LBA range to an extent range of the plurality of extent identifiers.
9. The method of claim 1 , wherein a plurality of write operations includes the write operation and each of the plurality of write operations is tagged with the first stream group identifier, and the method includes:
issuing the plurality of writes to the solid state drive that stores write data from the plurality of write operations in a same allocation block.
10. The method of claim 1 , further comprising:
receiving second write activity information for the plurality of extents for a second sampling period subsequent to the first sampling period;
for each of the plurality of RAID groups, performing stream detection processing in accordance with the second write activity information and determining a second plurality of stream groups, wherein each of the second plurality of stream groups for each of the plurality of RAID groups includes extents of said each RAID group belonging to a same write stream in said each RAID group, wherein each of the second plurality of stream groups of each of the plurality of RAID groups is uniquely identified using one of the plurality of stream group identifiers;
tagging a second write operation directed to a second of the plurality of extents with a second of the plurality of stream group identifiers denoting a stream group including the second extent; and
issuing the second write operation to the solid state drive.
11. The method of claim 1 , wherein each of the plurality of extents assigned to each of the plurality of RAID groups is striped across all physical storage devices of said each RAID group.
12. A system comprising:
a processor; and
a memory comprising code stored thereon that, when executed, performs a method of processing I/O operations comprising:
randomly assigning a plurality of extent identifiers to a plurality of RAID (redundant array of independent disks) groups, wherein each of the plurality of extent identifiers denotes a different one of a plurality of extents;
receiving first write activity information for the plurality of extents for a first sampling period;
for each of the plurality of RAID groups, performing stream detection processing in accordance with the first write activity information and determining a plurality of stream groups, wherein each of the plurality of stream groups for each of the plurality of RAID groups includes extents of said each RAID group belonging to a same write stream in said each RAID group, wherein each of the plurality of stream groups of each of the plurality of RAID groups is uniquely identified using one of a plurality of stream group identifiers;
tagging a write operation directed to one of the plurality of extents with a first of the plurality of stream group identifiers denoting a stream group including the one extent; and
issuing the write operation to a solid state drive of one of the plurality of RAID groups.
13. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of processing I/O operations comprising:
randomly assigning a plurality of extent identifiers to a plurality of RAID (redundant array of independent disks) groups, wherein each of the plurality of extent identifiers denotes a different one of a plurality of extents;
receiving first write activity information for the plurality of extents for a first sampling period;
for each of the plurality of RAID groups, performing stream detection processing in accordance with the first write activity information and determining a plurality of stream groups, wherein each of the plurality of stream groups for each of the plurality of RAID groups includes extents of said each RAID group belonging to a same write stream in said each RAID group, wherein each of the plurality of stream groups of each of the plurality of RAID groups is uniquely identified using one of a plurality of stream group identifiers;
tagging a write operation directed to one of the plurality of extents with a first of the plurality of stream group identifiers denoting a stream group including the one extent; and
issuing the write operation to a solid state drive of one of the plurality of RAID groups.
14. The non-transitory computer readable medium of claim 13 , wherein said stream detection processing for a first of the plurality of RAID groups includes:
determining, using a clustering algorithm and in accordance with the first write activity information, a plurality of clusters of extents of the first RAID group, wherein each of the plurality of clusters of extents of the first RAID group identifies a different write stream of extents of the first RAID group.
15. The non-transitory computer readable medium of claim 14 , wherein the clustering algorithm is a k-means clustering algorithm.
16. The non-transitory computer readable medium of claim 14 , wherein each of the plurality of clusters of extents of the first RAID group includes extents of the first RAID group determined by the clustering algorithm as similar with respect to a plurality of dimensions.
17. The non-transitory computer readable medium of claim 16 , wherein the plurality of dimensions include extent identifier and write activity.
18. The non-transitory computer readable medium of claim 17 , wherein each of the plurality of clusters of extents of the first RAID group includes extents of the first RAID group determined by the clustering algorithm to have similar write activity and also similar locality of reference based on extent identifiers associated with extents of said each cluster.
19. The non-transitory computer readable medium of claim 18 , wherein the first write activity information for each of the plurality of extents includes an amount of data written to said each extent per unit of time during the first sampling period.
20. The non-transitory computer readable medium of claim 14 , wherein the method includes:
forming an aggregate LBA (logical block address range) across contiguous LBA ranges of a plurality of logical devices; and
sequentially mapping portions of the aggregate LBA range to an extent range of the plurality of extent identifiers.