Programmatically mapping workloads to networked object storage
Disclosed are techniques for programmatically mapping workloads for storage in different classes of a networked computer data storage environment, using a set of rules or data storage hints and properties of the workloads themselves. With the set of data storage hints, metadata sets are assigned to files of the incoming workloads to the networked computer data storage environment based on the properties of the workloads. Then, a target storage class is determined for each file of the workload from the plurality of storage classes present in the networked computer data storage environment. The workload files are then mapped to a logical bucket that is associated with the plurality of storage classes through a shared namespace, with the assigned metadata set for a file cueing the filesystem modules as to which storage class within the logical bucket to direct the various files of the workload to.
1. A computer-implemented method (CIM) comprising:
receiving a first file-type based rule with the rule being that if a new file has the name “TMP” then the new file will not be replicated to network storage;
receiving a second file-type based rule with the rule being that if a new file has a size less than one gigabyte then the file will not be compressed when it is replicated to network storage;
receiving a third file-type based rule with the rule being that if a new file has been opened and closed in less than one millisecond then caching to network storage is started;
receiving a first output file to be stored in a first storage location and replicated in network storage from a computing workload being performed by a cloud computing system;
determining that the first output file is named “TMP”;
responsive to the determination that the first output file is named “TMP,” applying the first file-type based rule to prevent replication to network storage;
receiving a second output file to be stored in the first storage location and stored in network storage from a computing workload being performed by a cloud computing system;
determining that the second output file has a size of less than one gigabyte;
responsive to the determination that the second output file has a size of less than one gigabyte, applying the second file-type based rule to prevent compression of the second output file when it is replicated to network storage;
receiving a third output file to be stored in a first storage location and stored in network storage from a computing workload being performed by a cloud computing system;
determining that the third output file has been closed in less than one millisecond;
responsive to the determination that the third output file has been closed in less than one millisecond, applying the third file-type based rule to start caching the third output file for replication to network storage.