Managing storage system performance
Implementations are provided herein for systems, methods, and a non-transitory computer product configured to analyze an input/output (IO) pattern for a data storage system, to identify an application type based on the IO pattern, and to select optimal deduplication and compression configurations based on the application type. The teachings herein facilitate machine learning of various metrics and the interrelations between these metrics, such as past IO patterns, application types, deduplication configurations, compression configurations, and overall system performance. These metrics and interrelations can be stored in a data lake. In some embodiments, data objects can be segmented in order to optimize configurations with more granularity. In additional embodiments, predictive techniques are used to select deduplication and compression configurations.
1. A method comprising:
analyzing input/output (I/O) operations directed to a data storage system by one or more host devices;
determining at least one I/O pattern for the I/O operations based at least in part on the analyzing;
identifying one or more application types for respective ones of the one or more determined I/O patterns;
evaluating at least one of a preliminary deduplication configuration and a preliminary compression configuration by predicting one or more future performance characteristics of the data storage system; and
selecting at least one of a particular deduplication configuration and a particular compression configuration for use in the data storage system for data associated with the I/O operations based at least in part on the predicted one or more future performance characteristics and the identified one or more application types.
2. The method of claim 1 , wherein the data associated with the I/O operations comprises at least one storage object targeted by the I/O operations.
3. The method of claim 1 , further comprising deduplicating the data in the data storage system in accordance with the selected particular deduplication configuration.
4. The method of claim 1 , further comprising compressing the data in the data storage system in accordance with the selected particular compression configuration.
5. The method of claim 1 , wherein selecting at least one of a particular deduplication configuration and a particular compression configuration is performed in the data storage system and/or in an associated management system, but external to the one or more host devices.
6. The method of claim 1 , wherein determining at least one I/O pattern for the I/O operations based at least in part on the analyzing comprises determining at least one I/O pattern based at least in part on read-write splits in the I/O operations.
7. The method of claim 1 , wherein determining at least one I/O pattern for the I/O operations based at least in part on the analyzing comprises determining at least one I/O pattern based at least in part on sequential-random operation splits in the I/O operations.
8. The method of claim 1 , wherein determining at least one I/O pattern for the I/O operations based at least in part on the analyzing comprises applying a machine learning process to identify at least one I/O pattern for the I/O operations utilizing information characterizing a plurality of previously-determined I/O patterns.
9. The method of claim 1 , further comprising:
computing I/O processing performance statistics for the data storage system utilizing the selected at least one of the particular deduplication configuration and the particular compression configuration; and
modifying at least one of the particular deduplication configuration and the particular compression configuration for data associated with the I/O operations based at least in part on the computed I/O processing performance statistics.
10. The method of claim 1 , wherein the application type is chosen from a catalog of application types, the catalog of application types being compiled by a machine learning module.
11. The method of claim 1 , wherein selecting at least one of a particular deduplication configuration and a particular compression configuration is further based on an application I/O rate and an overall array load.
12. The method of claim 2 , wherein the storage object is split into a first data block and a second data block, the method further comprising:
analyzing a first I/O pattern for the first data block;
analyzing a second I/O pattern for the second data block;
identifying a first application type based on the first I/O pattern;
identifying a second application type based on the second I/O pattern;
selecting at least one of a first deduplication configuration and a first compression configuration based on the first application type; and
selecting at least one of a second deduplication configuration and a second compression configuration based on the second application type.
13. The method of claim 1 , wherein selecting at least one of a particular deduplication configuration and a particular compression configuration further comprises prior to evaluating the at least one of a preliminary deduplication configuration and a preliminary compression configuration:
selecting the at least one of a preliminary deduplication configuration and a preliminary compression configuration based on the application type.
14. A system comprising:
at least one processing device comprising a processor and a memory;
said at least one processing device being configured:
to analyze input/output (I/O) operations directed to a data storage system by one or more host devices;
to determine at least one I/O pattern for the I/O operations based at least in part on the analyzing;
to identify one or more application types for respective ones of the one or more determined I/O patterns;
to evaluate at least one of a preliminary deduplication configuration and a preliminary compression configuration by predicting one or more future performance characteristics of the data storage system; and
to select at least one of a particular deduplication configuration and a particular compression configuration for use in the data storage system for data associated with the I/O operations based at least in part on the predicted one or more future performance characteristics and the identified one or more application types.
15. The system of claim 14 , wherein selecting at least one of a particular deduplication configuration and a particular compression configuration is performed in the data storage system and/or in an associated management system, but external to the one or more host devices.
16. The system of claim 14 , wherein determining at least one I/O pattern for the I/O operations based at least in part on the analyzing comprises applying a machine learning process to identify at least one I/O pattern for the I/O operations utilizing information characterizing a plurality of previously-determined I/O patterns.
17. The system of claim 14 , being further configured:
to compute I/O processing performance statistics for the data storage system utilizing the selected at least one of the particular deduplication configuration and the particular compression configuration; and
to modify at least one of the particular deduplication configuration and the particular compression configuration for data associated with the I/O operations based at least in part on the computed I/O processing performance statistics.
18. A non-transitory computer readable medium with program instructions stored thereon, the program instructions when executed by at least one processing device comprising a processor coupled to a memory causing said at least one processing device:
to analyze input/output (I/O) operations directed to a data storage system by one or more host devices;
to determine at least one I/O pattern for the I/O operations based at least in part on the analyzing;
to identify one or more application types for respective ones of the one or more determined I/O patterns;
to evaluate at least one of a preliminary deduplication configuration and a preliminary compression configuration by predicting one or more future performance characteristics of the data storage system; and
to select at least one of a particular deduplication configuration and a particular compression configuration for use in the data storage system for data associated with the I/O operations based at least in part on the predicted one or more future performance characteristics and the identified one or more application types.
19. The non-transitory computer readable medium of claim 18 , wherein selecting at least one of a particular deduplication configuration and a particular compression configuration is performed in the data storage system and/or in an associated management system, but external to the one or more host devices.
20. The non-transitory computer readable medium of claim 18 , wherein determining at least one I/O pattern for the I/O operations based at least in part on the analyzing comprises applying a machine learning process to identify at least one I/O pattern for the I/O operations utilizing information characterizing a plurality of previously-determined I/O patterns.