Process aware data compression
Determining an expected compression rate for a prospective process in a federated system includes obtaining compression rate data for existing processes in the federated system, compiling the compression rate data into a plurality of entries in a process name table according to process identifier, client, and industry, determining a specific entry in the process name table for an existing process that most closely matches the prospective process, and determining an expected compression rate of the prospective process based on the compression rate data for the specific entry. Compression rate data may be provided by a driver at host systems that sends compression rate information to a central repository. The central repository may be provided by a host system at a data center of the federated system. The compression rate data may use a sliding average that weighs the data more heavily to favor more recent data.
1. A method of determining an expected compression rate for a prospective process in a federated system including a plurality of host systems, comprising:
obtaining compression rate data for existing processes in the federated system;
compiling the compression rate data into a plurality of entries in a process name table according to process identifier, client, and industry, each entry specifying a process identifier of a process executing on one of the host systems and a client who owns the process, and specifying a compression rate indicating an amount of data compression achieved when the process writes data from the host system to a storage device on the federated system;
determining a specific entry in the process name table for an existing process that most closely matches the prospective process; and
determining an expected compression rate of the prospective process based on the compression rate data for the specific entry.
2. A method, according to claim 1 , wherein compression rate data is provided by a driver at host systems that sends compression rate information to a central repository.
3. A method, according to claim 2 , wherein the central repository is provided by a host system at a data center of the federated system.
4. A method, according to claim 1 , wherein matching an entry in the process name table includes matching a process identifier and matching a client.
5. A method, according to claim 4 , wherein, in response to there being no matching process, choosing an entry in the process name table having a closest process to the prospective process.
6. A method, according to claim 1 , wherein matching an entry in the process name table includes matching a process identifier and matching an industry.
7. A method, according to claim 6 , wherein, in response to there being no matching industry, choosing an entry in the process name table having a closest industry to that of the prospective process.
8. A method, according to claim 1 , wherein the compression rate data uses a sliding average that weighs the data more heavily to favor more recent data.
9. A method, according to claim 1 , wherein the compression rate data is provided as histogram data.
10. A method, according to claim 9 , wherein the expected compression rate is based on a mode of histogram data.
11. A non-transitory computer readable medium containing software that determines an expected compression rate for a prospective process in a federated system including a plurality of host systems, the software comprising:
executable code that obtains compression rate data for existing processes in the federated system;
executable code that compiles the compression rate data into a plurality of entries in a process name table according to process identifier, client, and industry, each entry specifying a process identifier of a process executing on one of the host systems and a client who owns the process, and specifying a compression rate indicating an amount of data compression achieved when the process writes data from the host system to a storage device on the federated system;
executable code that determines a specific entry in the process name table for an existing process that most closely matches the prospective process; and
executable code that determines an expected compression rate of the prospective process based on the compression rate data for the specific entry.
12. A non-transitory computer readable medium, according to claim 11 , wherein compression rate data is provided by a driver at host systems that sends compression rate information to a central repository.
13. A non-transitory computer readable medium, according to claim 12 , wherein the central repository is provided by a host system at a data center of the federated system.
14. A non-transitory computer readable medium, according to claim 11 , wherein matching an entry in the process name table includes matching a process identifier and matching a client.
15. A non-transitory computer readable medium, according to claim 14 , wherein, in response to there being no matching process, choosing an entry in the process name table having a closest process to the prospective process.
16. A non-transitory computer readable medium, according to claim 11 , wherein matching an entry in the process name table includes matching a process identifier and matching an industry.
17. A non-transitory computer readable medium, according to claim 16 , wherein, in response to there being no matching industry, choosing an entry in the process name table having a closest industry to that of the prospective process.
18. A non-transitory computer readable medium, according to claim 11 , wherein the compression rate data uses a sliding average that weighs the data more heavily to favor more recent data.
19. A non-transitory computer readable medium, according to claim 11 , wherein the compression rate data is provided as histogram data.
20. A non-transitory computer readable medium, according to claim 19 , wherein the expected compression rate is based on a mode of histogram data.