System and method for a content-aware and context-aware compression algorithm selection model for a file system
A method for managing a file system includes obtaining, by a compression optimizing manager, a compression algorithm selection request for the file system, determining a set of selection inputs based on a set of file system parameters of the file system, applying a compression selection model to the set of selection inputs to obtain a compression algorithm selection, and initiating a file system compression implementation of the file system using the compression algorithm selection.
1. A method for managing a file system, the method comprising:
obtaining, by a compression optimizing manager, a compression algorithm selection request for the file system;
determining a set of selection inputs based on a set of file system parameters of the file system,
wherein determining the set of selection inputs comprises:
performing a file system parameter analysis to determine a uniqueness of each of the set of file system parameters of the file system relative to other file systems;
making a determination that a portion of the set of file system parameters of the file system exceeds a uniqueness threshold, and
based on the determination: assigning the portion of the set of file system parameters of the file system to the set of selection inputs,
wherein the set of selection inputs comprise a subset of the file system parameters preferred for evaluation, and
wherein the set of file system parameters comprises: a file system protocol of the file system, a geographical location of a production host implementing the file system, and a geographical location of a target storage device in which the file system is to be stored after compression;
applying a compression selection model to the set of selection inputs to obtain a compression algorithm selection; and
initiating a file system compression implementation of the file system using the compression algorithm selection.
2. The method of claim 1 , further comprising:
prior to obtaining the compression algorithm selection request:
identifying a set of compression algorithms;
identifying the set of file system parameters of the file system;
obtaining a training file system;
performing a compression algorithm analysis on each compression algorithm in the set of compression algorithms using the training file system to obtain a set of training compression algorithm performance metrics; and
generating the compression selection model using the set of training compression algorithm performance metrics and the set of file system parameters of the file system.
3. The method of claim 2 , wherein the set of file system parameters of the file system further comprises at least one of: a type of the production host, a file size, a file format, a number of files in the file system, and a backup storage type.
4. The method of claim 1 , wherein determining the set of selection inputs comprises:
sending the set of file system parameters to a client; and
obtaining a response that specifies the set of selection inputs.
5. The method of claim 1 , wherein determining the set of selection inputs comprises:
sending a selection request to a client that specifies selecting at least a portion of the set of file system parameters of the file system;
obtaining a response based on the selection request that specifies the portion of the set of file system parameters of the file system; and
based on the response, assigning the portion of the set of file system parameters of the file system to the set of selection inputs.
6. The method of claim 1 , further comprising:
after initiating the file system compression implementation:
obtaining an update to a portion of the file system;
initiating a compression of the portion of the file system using the compression algorithm selection;
performing a performance analysis on the compression of the portion of the file system to obtain a set of compression algorithm performance metrics; and
updating the compression selection model using the set of compression algorithm performance metrics and the compression algorithm selection.
7. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for managing the file system, the method comprising:
obtaining, by a compression optimizing manager, a compression algorithm selection request for the file system;
determining a set of selection inputs based on a set of file system parameters of the file system,
wherein determining the set of selection inputs comprises:
performing a file system parameter analysis to determine a uniqueness of each of the set of file system parameters of the file system relative to other file systems;
making a determination that a portion of the set of file system parameters of the file system exceeds a uniqueness threshold; and
based on the determination: assigning the portion of the set of file system parameters of the file system to the set of selection inputs,
wherein the set of selection inputs comprise a subset of the file system parameters preferred for evaluation, and
wherein the set of file system parameters comprises: a file system protocol of the file system, a geographical location of a production host implementing the file system, and a geographical location of a target storage device in which the file system is to be stored after compression;
applying a compression selection model to the set of selection inputs to obtain a compression algorithm selection; and
initiating a file system compression implementation of the file system using the compression algorithm selection.
8. The non-transitory computer readable medium of claim 7 , further comprising:
prior to obtaining the compression algorithm selection request:
identifying a set of compression algorithms;
identifying the set of file system parameters of the file system;
obtaining a training file system;
performing a compression algorithm analysis on each compression algorithm in the set of compression algorithms using the training file system to obtain a set of training compression algorithm performance metrics; and
generating the compression selection model using the set of training compression algorithm performance metrics and the set of file system parameters of the file system.
9. The non-transitory computer readable medium of claim 8 , wherein the file system parameters further comprises at least one of: a type of the production host, a file size, a file format, a number of files in the file system, and a backup storage type.
10. The non-transitory computer readable medium of claim 7 , wherein determining the set of selection inputs comprises:
sending the set of file system parameters to a client; and
obtaining a response that specifies the set of selection inputs.
11. The non-transitory computer readable medium of claim 7 , wherein determining the set of selection inputs comprises:
performing a file system parameter analysis to determine a uniqueness of each of the set of file system parameters of the file system;
making a determination that a portion of the file system parameters exceeds a uniqueness threshold; and
based on the determination: assigning the portion of the set of file system parameters of the file system to the set of selection inputs.
12. The non-transitory computer readable medium of claim 7 , further comprising:
after initiating the file system compression implementation:
obtaining an update to a portion of the file system;
initiating a compression of the portion of the file system using the compression algorithm selection;
performing a performance analysis on the compression of the portion of the file system to obtain a set of compression algorithm performance metrics; and
updating the compression selection model using the set of compression algorithm performance metrics and the compression algorithm selection.
13. A system comprising:
a processor; and
memory comprising instructions, which when executed by the processor, perform a method comprising:
obtaining, by a compression optimizing manager, a compression algorithm selection request for the file system;
determining a set of selection inputs based on a set of file system parameters of the file system,
wherein determining the set of selection inputs comprises:
performing a file system parameter analysis to determine a uniqueness of each of the set of file system parameters of the file system relative to other file systems;
making a determination that a portion of the set of file system parameters of the file system exceeds a uniqueness threshold; and
based on the determination: assigning the portion of the set of file system parameters of the file system to the set of selection inputs,
wherein the set of selection inputs comprise a subset of the file system parameters preferred for evaluation, and
wherein the set of file system parameters comprises: a file system protocol of the file system, a geographical location of a production host implementing the file system, and a geographical location of a target storage device in which the file system is to be stored after compression;
applying a compression selection model to the set of selection inputs to obtain a compression algorithm selection; and
initiating a file system compression implementation of the file system using the compression algorithm selection.
14. The system of claim 13 , further comprising:
prior to obtaining the compression algorithm selection request:
identifying a set of compression algorithms;
identifying the set of file system parameters of the file system;
obtaining a training file system;
performing a compression algorithm analysis on each compression algorithm in the set of compression algorithms using the training file system to obtain a set of training compression algorithm performance metrics; and
generating the compression selection model using the set of training compression algorithm performance metrics and the set of file system parameters of the file system.
15. The system of claim 14 , wherein the set of file system parameters of the file system further comprises at least one of: a type of the production host, a file size, a file format, a number of files in the file system, and a backup storage type.
16. The system of claim 13 , wherein determining the set of selection inputs comprises:
sending the set of file system parameters to a client; and
obtaining a response that specifies the set of selection inputs.
17. The system of claim 13 , wherein determining the set of selection inputs comprises:
performing a file system parameter analysis to determine a uniqueness of each of the set of file system parameters of the file system;
making a determination that a portion of the file system parameters exceeds a uniqueness threshold; and
based on the determination: assigning the portion of the set of file system parameters of the file system to the set of selection inputs.