Shared drive storage stack distributed QoS method and system
A data storage system and method comprising: at least two servers that comprise a storage volume and configured to run an operating system designated to host data accessible and exposable over a data plane (DP) network, at least one orchestrator configured to interact with each of said servers and designated to control a control plane (CP) of said DP network, wherein the at least two storage volumes create a distributed storage stack, wherein each server is configured to manage and enforce local QoS DP originated from its own volume, and wherein the orchestrator is configured to centrally orchestrate and enforce QoS CP limitations within the distributed storage stack.
1. A data storage system, comprising:
(i) at least two servers that each comprise a storage volume and configured to run an operating system designated to host data accessible and exposable over a data plane (DP) network, and
(ii) at least one orchestrator configured to interact with each of said servers and designated to control a control plane (CP) of said DP network, wherein the at least two storage volumes create a distributed storage stack,
wherein each server is configured to manage and enforce local quality of service (QoS) DP originated from its own volume, and
wherein the orchestrator is configured to centrally orchestrate and enforce QoS CP limitations within the distributed storage stack.
2. The system of claim 1 , wherein the orchestrator is configured calculate the required QoS of each volume.
3. The system of claim 1 , wherein the orchestrator is configured to allocate new hot spare disk from a storage pool in case of failure.
4. The system of claim 1 , wherein the orchestrator is configured to instruct each server to add a selected spare storage media to each storage volume.
5. The system of claim 1 , wherein the orchestrator is configured to add a selected spare disk to the shared storage volume.
6. The system of claim 1 , wherein the orchestrator is configured to instruct each server to dismiss a failed disk in each shared storage volume and rebuilding the new spare drive chunk with valid data.
7. The system of claim 1 , wherein the connection between the servers is conducted using a multipath connection.
8. The system of claim 1 , wherein the orchestrator is configured to provide alerts and handle failures for the distributed block device connection between servers.
9. The system of claim 1 , wherein multiple initiators are configured to be created per network path to each server.
10. The system of claim 1 , wherein the coordination between the servers is conducted using a CP orchestrator installed on each server.
11. The system of claim 1 , wherein the communication between the servers and the orchestrator is done using a designated software component installed on each of said servers.
12. The system of claim 1 , wherein the server is configured to utilize a redundant array of independent disks (RAID) storage stack component (SSC) configured to provide data redundancy originated from multiple designated portions of the storage volume at multiple servers.
13. The system of claim 12 , wherein the RAID SSC is configured to provide data redundancy originated from combined multiple initiator paths originated from the designated portion of the storage volume of at least two servers.
14. The system of claim 12 , wherein the calculation at the base of the orchestration process is based on a RAID schema.
15. The system of claim 13 , wherein the servers are located at different locations and wherein the orchestration is allocated across different resiliency domains.
16. The system of claim 15 , wherein the orchestration allocated while considering different resiliency domains also considers maintaining system balance.
17. The system of claim 1 , wherein the orchestrator is configured to interact with the servers using an administration protocol.
18. The system of claim 1 , wherein a designated portion of the storage volume is allocated using a logical volume manager (LVM) SSC.
19. The system of claim 1 , wherein the operations on each server may be implemented, wholly or partially, by a data processing unit (DPU).
20. A method for orchestrating and storing data, comprising the steps of:
(i) enabling at least two servers that each comprise a storage volume and configured to run an operating system to host data accessible and exposable over a data plane (DP) network, and
(ii) enabling at least one orchestrator to interact with each of said servers to control a control plane (CP) of said DP network, wherein the at least two storage volumes create a distributed storage stack, wherein each server is configured to manage and enforce local quality of service (QoS) DP originated from its own volume, and wherein the orchestrator is configured to centrally orchestrate and enforce QoS CP limitations within the distributed storage stack.