Dynamic erasure coding
Dynamic erasure coding for computing and data storage systems. A method embodiment commences upon accessing a set of fault tolerance policy attributes associated with the computing and data storage system. The topology of the system is analyzed to form mappings between the computing nodes of the system and the availability domains of the system. Based on the fault tolerance policy attributes, the topology, and the generated mapping, a plurality of feasible erasure coding configurations are generated. The feasible erasure coding configurations are scored. One or more high-scoring feasible erasure coding configurations are selected and deployed to the computing and data storage system. The method is repeated when there is a change in the fault tolerance policy attributes or in the topology. Depending on the topology and/or the nature of a change in the topology, more than one erasure coding configurations can be deployed onto the computing and data storage system.
1. A method comprising:
accessing a fault tolerance policy attribute describing a fault tolerance policy associated with a computing system;
accessing a computing node topology attribute that describes a location of a computing node of the computing system;
generating a plurality of candidate erasure coding configurations, wherein the candidate erasure coding configurations are generated based at least in part on at least one of, the fault tolerance policy attribute or the computing node topology attribute;
computing respective configuration scores corresponding to respective candidate erasure coding configurations of the plurality of candidate erasure coding configurations; and
selecting an erasure coding configuration from the plurality of candidate erasure coding configurations, the selecting being based at least in part on the respective configuration scores.
2. The method of claim 1 , further comprising:
identifying at least one deployed erasure coding configuration, wherein the deployed erasure coding configuration is different from the selected erasure coding configuration; and
converting the deployed erasure coding configuration to the selected erasure coding configuration.
3. The method of claim 1 , further comprising:
implementing a first selected erasure coding configuration to a first set of availability domains; and
implementing a second selected erasure coding configuration to a second set of availability domains.
4. The method of claim 1 , wherein generating the candidate erasure coding configurations is responsive to an erasure coding configuration event.
5. The method of claim 4 , wherein the erasure coding configuration event is triggered by at least one of, a user input, a policy change, or a topology change.
6. The method of claim 1 , wherein a constraint is derived from at least one of the fault tolerance policy attribute or the computing node topology attribute, and wherein the candidate erasure coding configurations are generated subject to the constraint.
7. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes a set of acts, the set of acts comprising:
accessing a fault tolerance policy attribute describing a fault tolerance policy associated with a computing system;
accessing a computing node topology attribute that describes a location of a computing node of the computing system;
generating a plurality of candidate erasure coding configurations, wherein the candidate erasure coding configurations are generated based at least in part on at least one of, the fault tolerance policy attribute or the computing node topology attribute;
computing respective configuration scores corresponding to respective candidate erasure coding configurations of the plurality of candidate erasure coding configurations; and
selecting an erasure coding configuration from the plurality of candidate erasure coding configurations, the selecting being based at least in part on the respective configuration scores.
8. The computer readable medium of claim 7 , wherein the respective configuration scores describe a quantitative relationship with at least one of, a performance objective, or a storage space savings objective.
9. The computer readable medium of claim 7 , wherein the fault tolerance policy attribute characterizes at least one of, a fault tolerance value, a replication factor, an availability domain awareness setting, a computing node affinity, or an erasure coding objective.
10. The computer readable medium of claim 7 , wherein the computing node topology attribute characterizes at least one of, a node identifier, an availability domain identifier, a site identifier, a host identifier, an IP address, or a node state.
11. The computer readable medium of claim 7 , wherein a plurality of computing node topology attributes describe an availability domain.
12. The computer readable medium of claim 11 , wherein the plurality of computing node topology attributes, provide a mapping between the availability domain and computing nodes.
13. The computer readable medium of claim 7 , wherein an availability domain corresponds to the computing node, a block, a host, a site, an appliance, a rack, or a data center.
14. The computer readable medium of claim 7 , wherein the set of acts further comprise:
identifying at least one deployed erasure coding configuration, wherein the deployed erasure coding configuration is different from the selected erasure coding configuration; and
converting the deployed erasure coding configuration to the selected erasure coding configuration.
15. The computer readable medium of claim 7 , wherein the set of acts further comprise:
implementing a first selected erasure coding configuration to a first set of availability domains; and
implementing a second selected erasure coding configuration to a second set of availability domains.
16. The computer readable medium of claim 7 , wherein generating the candidate erasure coding configurations is responsive to an erasure coding configuration event.
17. The computer readable medium of claim 16 , wherein the erasure coding configuration event is triggered by at least one of, a user input, a policy change, or a topology change.
18. The computer readable medium of claim 7 , wherein a constraint is derived from at least one of the fault tolerance policy attribute or the computing node topology attribute, and wherein the candidate erasure coding configurations are generated subject to the constraint.
19. A system comprising:
a storage medium having stored thereon a sequence of instructions; and
a processor that executes the sequence of instructions to cause a set of acts, the set of acts comprising,
accessing a fault tolerance policy attribute describing a fault tolerance policy associated with a computing system;
accessing a computing node topology attribute that describes a location of a computing node of the computing system;
generating a plurality of candidate erasure coding configurations, wherein the candidate erasure coding configurations are generated based at least in part on at least one of, the fault tolerance policy attribute or the computing node topology attribute;
computing respective configuration scores corresponding to respective candidate erasure coding configurations of the plurality of candidate erasure coding configurations; and
selecting an erasure coding configuration from the plurality of candidate erasure coding configurations, the selecting being based at least in part on the respective configuration scores.
20. The system of claim 19 , wherein a constraint is derived from at least one of the fault tolerance policy attribute or the computing node topology attribute, and wherein the candidate erasure coding configurations are generated subject to the constraint.