Methods, systems, and computer readable mediums for managing fault tolerance of hardware storage nodes
Methods, systems, and computer readable mediums for managing fault tolerance. A method includes receiving a request to establish a pool of data storage for an application of a distributed computing system. The distributed computing system includes hardware storage nodes integrated with compute nodes. The method includes receiving a target level of fault tolerance for the pool of data storage. The method includes establishing the pool of data storage by specifying, for each hardware storage node, a mirror hardware storage node for mirroring data stored on the hardware storage node so that the hardware storage node and the mirror hardware storage node do not share one or more pieces of physical equipment as specified in a physical layout of the hardware storage nodes to meet the target level of fault tolerance.
1. A method for managing fault tolerance, the method comprising:
receiving, by one or more computers, a request to establish a pool of data storage for an application of a distributed computing system, the distributed computing system comprising a plurality of hardware storage nodes each integrated with a respective compute node, wherein the distributed computing system comprises a hyper-converged system comprising a plurality of hyper-converged nodes, wherein the hyper-converged system is configured to perform virtualization and includes a hyper-converged storage manager configured to present the pool of data storage to the application as a single logical storage volume;
receiving, by the one or more computers, a target level of fault tolerance for the pool of data storage;
determining a physical layout of the hardware storage nodes by querying the hyper-converged nodes;
establishing, by the one or more computers, the pool of data storage by specifying, for each hardware storage node, a mirror hardware storage node for mirroring data stored on the hardware storage node so that the hardware storage node and the mirror hardware storage node do not share one or more pieces of physical equipment as specified in the physical layout of the hardware storage nodes to meet the target level of fault tolerance; and
auditing the pool of data storage by determining, using the physical layout of the hardware storage nodes, whether any of the hardware storage nodes shares a piece of physical equipment with a mirror hardware storage node in violation of the target level of fault tolerance and generating a list of hardware storage nodes that do not meet the target level of fault tolerance for the pool of data storage;
wherein the distributed computing system comprises a plurality of equipment racks housing the hardware storage nodes in a plurality of chassis, wherein the physical layout of the hardware storage nodes specifies a chassis identifier for each hardware storage node indicating which chassis houses the hardware storage node and a rack identifier for each hardware storage node indicating which equipment rack houses the hardware storage node, and wherein specifying a mirror hardware storage node for each hardware storage node comprises finding, for each hardware storage node, an unallocated hardware storage node having a chassis identifier different from the chassis identifier of the hardware storage node and a rack identifier different from the rack identifier of the hardware storage node, thereby achieving chassis-level and rack-level fault tolerance for the pool of data storage.
2. A system comprising:
one or more physical computers; and
a virtual storage aggregator implemented on the one or more physical computers for performing operations comprising:
receiving a request to establish a pool of data storage for an application of a distributed computing system, the distributed computing system comprising a plurality of hardware storage nodes each integrated with a respective compute node, wherein the distributed computing system comprises a hyper-converged system comprising a plurality of hyper-converged nodes, wherein the hyper-converged system is configured to perform virtualization and includes a hyper-converged storage manager configured to present the pool of data storage to the application as a single logical storage volume;
receiving a target level of fault tolerance for the pool of data storage;
determining a physical layout of the hardware storage nodes by querying the hyper-converged nodes;
establishing the pool of data storage by specifying, for each hardware storage node, a mirror hardware storage node for mirroring data stored on the hardware storage node so that the hardware storage node and the mirror hardware storage node do not share one or more pieces of physical equipment as specified in the physical layout of the hardware storage nodes to meet the target level of fault tolerance; and
auditing the pool of data storage by determining, using the physical layout of the hardware storage nodes, whether any of the hardware storage nodes shares a piece of physical equipment with a mirror hardware storage node in violation of the target level of fault tolerance and generating a list of hardware storage nodes that do not meet the target level of fault tolerance for the pool of data storage;
wherein the distributed computing system comprises a plurality of equipment racks housing the hardware storage nodes in a plurality of chassis, wherein the physical layout of the hardware storage nodes specifies a chassis identifier for each hardware storage node indicating which chassis houses the hardware storage node and a rack identifier for each hardware storage node indicating which equipment rack houses the hardware storage node, and wherein specifying a mirror hardware storage node for each hardware storage node comprises finding, for each hardware storage node, an unallocated hardware storage node having a chassis identifier different from the chassis identifier of the hardware storage node and a rack identifier different from the rack identifier of the hardware storage node, thereby achieving chassis-level and rack-level fault tolerance for the pool of data storage.
3. A non-transitory computer readable medium having stored thereon executable instructions which, when executed by one or more physical computers, cause the one or more physical computers to perform operations comprising:
receiving a request to establish a pool of data storage for an application of a distributed computing system, the distributed computing system comprising a plurality of hardware storage nodes each integrated with a respective compute node, wherein the distributed computing system comprises a hyper-converged system comprising a plurality of hyper-converged nodes, wherein the hyper-converged system is configured to perform virtualization and includes a hyper-converged storage manager configured to present the pool of data storage to the application as a single logical storage volume;
receiving a target level of fault tolerance for the pool of data storage;
determining a physical layout of the hardware storage nodes by querying the hyper-converged nodes;
establishing the pool of data storage by specifying, for each hardware storage node, a mirror hardware storage node for mirroring data stored on the hardware storage node so that the hardware storage node and the mirror hardware storage node do not share one or more pieces of physical equipment as specified in the physical layout of the hardware storage nodes to meet the target level of fault tolerance; and
auditing the pool of data storage by determining, using the physical layout of the hardware storage nodes, whether any of the hardware storage nodes shares a piece of physical equipment with a mirror hardware storage node in violation of the target level of fault tolerance and generating a list of hardware storage nodes that do not meet the target level of fault tolerance for the pool of data storage;
wherein the distributed computing system comprises a plurality of equipment racks housing the hardware storage nodes in a plurality of chassis, wherein the physical layout of the hardware storage nodes specifies a chassis identifier for each hardware storage node indicating which chassis houses the hardware storage node and a rack identifier for each hardware storage node indicating which equipment rack houses the hardware storage node, and wherein specifying a mirror hardware storage node for each hardware storage node comprises finding, for each hardware storage node, an unallocated hardware storage node having a chassis identifier different from the chassis identifier of the hardware storage node and a rack identifier different from the rack identifier of the hardware storage node, thereby achieving chassis-level and rack-level fault tolerance for the pool of data storage.