Predictive performance analysis for file systems
Embodiments are directed to managing file systems over a network. A core specification that defines characteristics of a portion of a file system and parameters may be provided. The characteristics may include one or more storage device characteristics. And, the parameters may include one or more of a cluster size, or a protection level. File system models may be generated based on the core specification and the parameters. Each file system model may correspond to a file system that conforms to a core specification and specific values for the parameters. A simulation engine may be employed to provide simulation results based on the file system models. Deployment models may be generated based on the simulation results. The deployment models may be employed to provision file systems that conform to the file system performance characteristics associated with its corresponding deployment model.
1. A method for managing file systems over a network using one or more processors that execute instructions to perform actions, comprising:
generating a plurality of file system models based on storage constraint characteristics and configuration information in a file system for each physical core component associated with providing storage for the file system and one or more parameters of one or more portions of the file system, wherein each file system model corresponds to the file system having a particular value for the one or more parameters, and wherein one or more simulation results are provided for each of the file system models, and wherein the storage constraint characteristics include technical performance values determined for a manufacturer and a model for each physical storage device that is a core component associated with providing storage for the file system, and wherein the manufacturer and the model are employed to remotely retrieve the technical performance values from a database or other service;
employing a core specification to define one or more immutable features of the file system, wherein the core specification specifies a plurality of core components with one or more performance properties that enable operation of the one or more immutable features;
generating a plurality of deployment models for the plurality of file system models based on the one or more simulation results and each corresponding file system having the particular value for the one or more parameters, wherein each deployment model is associated with one or more file system performance characteristics; and
employing one or more file system performance requirements to select one or more of the plurality of deployment models to provision one or more file systems having the one or more file system performance characteristics associated with its corresponding deployment model based on the one or more file system performance characteristics including one or more of Mean Time to Data Loss (MTDL), protection level, recovery impact, or number of nodes and also based on one or more of storage device characteristics comprising one or more of annualized failure rate (AFR), mean time before failure (MTBF), cache information, capacity, data transfer speed, and power requirements.
2. The method of claim 1 , wherein the one or more parameters include one or more of node counts for a plurality of cluster sizes, or a plurality of protection levels.
3. The method of claim 1 , wherein generating the plurality of file system models further comprises:
providing the core specification that includes one or more characteristics of the one or more portions of the file system, wherein the one or more characteristics include one or more storage device characteristics; and
employing the core specification and the one or more parameters to generate the plurality of file system models.
4. The method of claim 1 , wherein providing the one or more simulation results, further comprises:
simulating performance of one or more clusters of storage devices having one or more node count sizes and one or more protection levels for each of the file system models.
5. The method of claim 1 , further comprising:
monitoring one or more metrics for the one or more provisioned file systems; and
in response to one or more metrics diverging from the one or more selected deployment models, generating one or more notifications to a user.
6. The method of claim 1 , wherein generating the plurality of file system models, further comprises:
generating one or more deterministic file system models or one or more probabilistic file system models.
7. The method of claim 1 , wherein generating the plurality of deployment models, further comprises:
determining one or more coefficients that correspond to a top ranked curve generated by one or more functions of the one or more simulation results, wherein the one or more coefficients are included in the plurality of deployment models; and
wherein the one or more coefficients and the one or more functions are employed to provide one or more portions of the information used to provision the one or more file systems.
8. A network computer for managing file systems over a network, comprising:
a memory that stores at least instructions; and
one or more processors that execute instructions that are configured to cause performance of actions, including:
generating a plurality of file system models based on storage constraint characteristics and configuration information in a file system for each physical core component associated with providing storage for the file system and one or more parameters of one or more portions of the file system, wherein each file system model corresponds to the file system having a particular value for the one or more parameters, and wherein one or more simulation results are provided for each of the file system models, and wherein the storage constraint characteristics include technical performance values determined for a manufacturer and a model for each physical storage device that is a core component associated with providing storage for the file system, and wherein the manufacturer and the model are employed to remotely retrieve the technical performance values from a database or other service;
employing a core specification to define one or more immutable features of the file system, wherein the core specification specifies a plurality of core components with one or more performance properties that enable operation of the one or more immutable features;
generating a plurality of deployment models for the plurality of file system models based on the one or more simulation results and each corresponding file system having the particular value for the one or more parameters, wherein each deployment model is associated with one or more file system performance characteristics; and
employing one or more file system performance requirements to select one or more of the plurality of deployment models to provision one or more file systems having the one or more file system performance characteristics associated with its corresponding deployment model based on the one or more file system performance characteristics including one or more of Mean Time to Data Loss (MTDL), protection level, recovery impact, or number of nodes and also based on one or more of storage device characteristics comprising one or more of annualized failure rate (AFR), mean time before failure (MTBF), cache information, capacity, data transfer speed, and power requirements.
9. The network computer of claim 8 , wherein the one or more parameters include one or more of node counts for a plurality of cluster sizes, or a plurality of protection levels.
10. The network computer of claim 8 , wherein generating the plurality of file system models further comprises:
providing a core specification that includes one or more characteristics of the one or more portions of the file system, wherein the one or more characteristics include one or more storage device characteristics; and
employing the core specification and the one or more parameters to generate the plurality of file system models.
11. The network computer of claim 8 , wherein providing the one or more simulation results, further comprises:
simulating performance of one or more clusters of storage devices having one or more node count sizes and one or more protection levels for each of the file system models.
12. The network computer of claim 8 , further comprising:
monitoring one or more metrics for the one or more provisioned file systems; and
in response to one or more metrics diverging from the one or more selected deployment models, generating one or more notifications to a user.
13. The network computer of claim 8 , wherein generating the plurality of file system models, further comprises:
generating one or more deterministic file system models or one or more probabilistic file system models.
14. The network computer of claim 8 , wherein generating the plurality of deployment models, further comprises:
determining one or more coefficients that correspond to a top ranked curve generated by one or more functions of the one or more simulation results, wherein the one or more coefficients are included in the plurality of deployment models; and
wherein the one or more coefficients and the one or more functions are employed to provide one or more portions of the information used to provision the one or more file systems.
15. A processor readable non-transitory storage media that includes instructions that are configured to cause performance of actions for managing a file system over a network, wherein execution of the instructions by one or more processors on one or more network computers performs actions, comprising:
generating a plurality of file system models based on storage constraint characteristics and configuration information in a file system for each physical core component associated with providing storage for the file system and one or more parameters of one or more portions of the file system, wherein each file system model corresponds to the file system having a particular value for the one or more parameters, and wherein one or more simulation results are provided for each of the file system models; and wherein the storage constraint characteristics include technical performance values determined for a manufacturer and a model for each physical storage device that is a core component associated with providing storage for the file system, and wherein the manufacturer and the model are employed to remotely retrieve the technical performance values from a database or other service;
employing a core specification to define one or more immutable features of the file system, wherein the core specification specifies a plurality of core components with one or more performance properties that enable operation of the one or more immutable features;
generating a plurality of deployment models for the plurality of file system models based on the one or more simulation results and each corresponding file system having the particular value for the one or more parameters, wherein each deployment model is associated with one or more file system performance characteristics; and
employing one or more file system performance requirements to select one or more of the plurality of deployment models to provision one or more file systems having the one or more file system performance characteristics associated with its corresponding deployment model based on the one or more file system performance characteristics including one or more of Mean Time to Data Loss (MTDL), protection level, recovery impact, or number of nodes and also based on one or more of storage device characteristics comprising one or more of annualized failure rate (AFR), mean time before failure (MTBF), cache information, capacity, data transfer speed, and power requirements.
16. The processor readable non-transitory storage media of claim 15 , wherein the one or more parameters include one or more of node counts for a plurality of cluster sizes, or a plurality of protection levels.
17. The processor readable non-transitory storage media of claim 15 , wherein generating the plurality of file system models further comprises:
providing a core specification that includes one or more characteristics of the one or more portions of the file system, wherein the one or more characteristics include one or more storage device characteristics; and
employing the core specification and the one or more parameters to generate the plurality of file system models.
18. The processor readable non-transitory storage media of claim 15 , wherein providing the one or more simulation results, further comprises:
simulating performance of one or more clusters of storage devices having one or more node count sizes and one or more protection levels for each of the file system models.
19. The processor readable non-transitory storage media of claim 15 , further comprising:
monitoring one or more metrics for the one or more provisioned file systems; and
in response to one or more metrics diverging from the one or more selected deployment models, generating one or more notifications to a user.
20. The processor readable non-transitory storage media of claim 15 , wherein generating the plurality of deployment models, further comprises:
determining one or more coefficients that correspond to a top ranked curve generated by one or more functions of the one or more simulation results, wherein the one or more coefficients are included in the plurality of deployment models; and
wherein the one or more coefficients and the one or more functions are employed to provide one or more portions of the information used to provision the one or more file systems.