Flexible data placement using log structured system

1 . A computer-implemented method comprising:

receiving, at a storage system, a plurality of write operations from a plurality of applications executing on one or more hosts;

recording, at the storage system, the plurality of write operations in a plurality of entries in a log; and

flushing, from the log, the plurality of entries denoting the plurality of write operations, wherein said flushing includes:

segregating first content written by the plurality of write operations from the plurality of applications into a plurality of streams, wherein each of the plurality of streams corresponds to one of a plurality of classes or types of data associated with one of a plurality of I/O profiles characterizing said each stream and said one class or type of data; and

for each of the plurality of streams corresponding to said one class or type of data associated with said one I/O profile, persistently storing a corresponding portion of the first content in one of a plurality of sub log structured systems (LSSs) used for only storing data of said one class or type that corresponds to said each stream;

wherein the plurality of sub LSSs correspond to classes or types of storage, wherein each of the classes or types of storage corresponds to one of the plurality of classes or types of data, and wherein the method includes:

determining that a first of the classes or types of storage needs first additional storage capacity in connection with storing data of a first of the plurality of classes or types of data; and

in response to determining that the first class or type of storage needs the first additional storage capacity, performing first processing to increase a first allocated storage capacity of the first class or type of storage.

2 . The computer-implemented method of claim 1 , wherein the plurality of sub LSSs each include back-end (BE) non-volatile storage of the storage system.

3 . The computer-implemented method of claim 1 , wherein, for each of the sub LSSs, each time updated content written to a logical address is stored in said each sub LSS whereby the updated content replaces existing content stored at the logical address, the updated content is written to a new physical location that is different from a current physical location of the existing content.

4 . The computer-implemented method of claim 1 , wherein each of the plurality of I/O profiles, which corresponds to a first of the plurality of classes or types of data and a first of the plurality of streams, includes one or more characteristics of said first class or type of data, wherein the one or more characteristics include any of: one or more I/O workload characteristics of the first class or type of data; a data deduplication ratio for the first class or type of data; a data compression ratio for the first class or type of data, and a tag identifying one or more of the plurality of applications that write data of the first class or type of data.

5 . The computer-implemented method of claim 4 , wherein the one or more I/O workload characteristics includes any of: an indication or whether a first I/O workload of the first class or type of data is write heavy whereby the first I/O workload has more write operations than read operations, an indication of whether writes of the first I/O workload exhibit spatial locality, and an indication of whether writes of the first I/O workload exhibit temporal locality.

6 . The computer-implemented method of claim 4 , wherein the plurality of applications writes to a plurality of storage entities each associated with one of the plurality of I/O profiles.

7 . The computer-implemented method of claim 6 , wherein the plurality of storage entities include any of: one or more volumes, and one or more file systems.

8 . The computer-implemented method of claim 1 , wherein the first class or type of storage is a target class, and wherein the first processing includes:

redistributing or reallocating first free storage capacity from a second of the classes or types of storage to the target class, wherein the second class or type of storage is a source class.

9 . The computer-implemented method of claim 8 , wherein said first processing includes:

performing first garbage collection (GC) processing on the source class to generate a first free storage segment reallocated from the source class to the target class.

10 . The computer-implemented method of claim 9 , wherein the first processing includes updating first information about the first free storage segment to indicate that the first free storage segment is included in the target class rather than the source class whereby the first free storage segment is used in connection with storing data of only a single class or type corresponding to the target class.

11 . The computer-implemented method of claim 10 , wherein the source class corresponds to a first sub LSS of the plurality of sub LSSs whereby the first GC processing is performed on storage of the first sub LSS to generate the first free storage segment of the first sub LSS, and wherein the first GC processing includes:

allocating a new storage segment associated with a new reclaim unit which is mapped to a first portion of BE non-volatile storage of the first sub LSS;

combining valid data from two storage segments of the first sub LSS to generate combined first data, wherein the two storage segments correspond to two reclaim units mapped to two portions of BE non-volatile storage of the first sub LSS;

storing the combined first data on the new claim unit;

deallocating a first segment of the two storage segments, wherein said deallocating includes unmapping the first segment corresponding to a first of the two reclaim units from a corresponding one of the two portions of BE non-volatile storage of the first sub LSS;

reallocating the first segment, wherein said reallocating includes mapping the first segment corresponding to the first reclaim unit to another corresponding portion of BE non-volatile storage of the first sub LSS; and

assigning the first segment to the target class corresponding to a second sub LSS of the plurality of sub LSSs, wherein the first segment is the first free storage segment.

12 . The computer-implemented method of claim 10 , wherein the source class corresponds to a first sub LSS of the plurality of sub LSSs whereby the first GC processing is performed on storage of the first sub LSS to generate the first free storage segment of the first sub LSS, and wherein the first GC processing includes:

combining valid data from two storage segments of the first sub LSS to generate combined first data, wherein the two storage segments are included in BE non-volatile storage of the first sub LSS;

storing the combined first data in a first of the two storage segments; and

reassigning a second of the two storage segments to the target class corresponding to a second sub LSS of the plurality of sub LSSs.

13 . The computer-implemented method of claim 12 , wherein the two storage segments are physical large blocks (PLBs) included in a first sub uber (SU) of BE non-volatile storage of the first sub LSS, wherein the first SU is the first free storage segment reallocated from the source class to the target class.

14 . The computer-implemented method of claim 13 , further comprising:

assigning the first SU to the target class corresponding to a second sub LSS of the plurality of sub LSSs, wherein the first SU is included in the second sub LSS after said assigning the first SU to the target class.

15 . The computer-implemented method of claim 1 , wherein the first class or type of storage corresponds to a first of the plurality of sub LSSs, wherein the first class or type of storage denotes a target class, wherein a second of the classes or types of storage corresponds to a second of the plurality of sub LSSs, wherein the second class denotes a source class, and wherein the method further comprises:

determining, based at least in part, on a first write amplification denoting write amplification of the plurality of sub LSSs, to allocate the first additional storage capacity to the target class; and

selecting the source class from the classes or types of storage based, at least in part, on a first priority of the target class and a second priority of the source class.

16 . The computer-implemented method of claim 15 , further comprising:

determining the first priority of the target class based, at least in part, on a first write density of the target class and a first write amplification of the target class; and

determining the second priority of the source class based, at least in part, on a second write density of the source class and a second write amplification of the source class, wherein the first priority of the target class indicates any one or more of: that the first write density of the target class is greater than the second write density of the source class, and that the first write amplification of the target class is greater than the second write amplification of the source class.

17 . The computer-implemented method of claim 1 , further comprising:

performing GC processing including:

for each of the plurality of sub LSSs, performing first GC processing on said each sub LSS using only storage of said each sub LSS, wherein said first GC processing includes generating a free segment or portion of storage of said each sub LSS.

18 . A system comprising:

one or more processors; and

one or more memories comprising code stored therein that, when executed, perform a method comprising:

receiving, at a storage system, a plurality of write operations from a plurality of applications executing on one or more hosts;

recording, at the storage system, the plurality of write operations in a plurality of entries in a log; and

flushing, from the log, the plurality of entries denoting the plurality of write operations, wherein said flushing includes:

segregating first content written by the plurality of write operations from the plurality of applications into a plurality of streams, wherein each of the plurality of streams corresponds to one of a plurality of classes or types of data associated with one of a plurality of I/O profiles characterizing said each stream and said one class or type of data; and

for each of the plurality of streams corresponding to said one class or type of data associated with said one I/O profile, persistently storing a corresponding portion of the first content in one of a plurality of sub log structured systems (LSSs) used for only storing data of said one class or type that corresponds to said each stream;

wherein the plurality of sub LSSs correspond to classes or types of storage, wherein each of the classes or types of storage corresponds to one of the plurality of classes or types of data, and wherein the method includes:

determining that a first of the classes or types of storage needs first additional storage capacity in connection with storing data of a first of the plurality of classes or types of data; and

in response to determining that the first class or type of storage needs the first additional storage capacity, performing first processing to increase a first allocated storage capacity of the first class or type of storage.

19 . One or more non-transitory computer readable media comprising code stored thereon that, when executed, performs a method comprising:

receiving, at a storage system, a plurality of write operations from a plurality of applications executing on one or more hosts;

recording, at the storage system, the plurality of write operations in a plurality of entries in a log; and

flushing, from the log, the plurality of entries denoting the plurality of write operations, wherein said flushing includes:

segregating first content written by the plurality of write operations from the plurality of applications into a plurality of streams, wherein each of the plurality of streams corresponds to one of a plurality of classes or types of data associated with one of a plurality of I/O profiles characterizing said each stream and said one class or type of data; and

for each of the plurality of streams corresponding to said one class or type of data associated with said one I/O profile, persistently storing a corresponding portion of the first content in one of a plurality of sub log structured systems (LSSs) used for only storing data of said one class or type that corresponds to said each stream;

wherein each of the plurality of I/O profiles, which corresponds to a first of the plurality of classes or types of data and a first of the plurality of streams, includes one or more characteristics of said first class or type of data, wherein the one or more characteristics include any of: one or more I/O workload characteristics of the first class or type of data; a data deduplication ratio for the first class or type of data; a data compression ratio for the first class or type of data, and a tag identifying one or more of the plurality of applications that write data of the first class or type of data; and

wherein the one or more I/O workload characteristics includes any of: an indication or whether a first I/O workload of the first class or type of data is write heavy whereby the first I/O workload has more write operations than read operations, an indication of whether writes of the first I/O workload exhibit spatial locality, and an indication of whether writes of the first I/O workload exhibit temporal locality.

20 . The one or more non-transitory computer readable media of claim 19 , wherein the plurality of sub LSSs correspond to classes or types of storage, wherein each of the classes or types of storage corresponds to one of the plurality of classes or types of data, and wherein the method includes:

determining that a first of the classes or types of storage needs first additional storage capacity in connection with storing data of a first of the plurality of classes or types of data; and

in response to determining that the first class or type of storage needs the first additional storage capacity, performing first processing to increase a first allocated storage capacity of the first class or type of storage.