Shared drive storage stack monitoring and recovery method and system
A data storage system and method comprising: at least one target server that comprises a storage media and configured to run an operating system designated to host data accessible over a data plane (DP) network, at least one initiator server comprises a storage media and configured to run an operating system designated to access and expose a remote resource/s over the data plane (DP) network, and at least one orchestrator configured to interact with each of said servers and designated to control a control plane (CP) of said DP network, wherein the combined storage medias form a shared drive storage stack, wherein the orchestrator is configured to utilize the CP in order to monitor and inspect the operability and condition of the shared drive storage stack, and further configured to monitor, record and inspect the proper operation of block device/s stored within said shared drive storage stack, wherein the orchestrator is configured to identify a malfunction affecting the shared drive storage stack, and wherein the orchestrator is configured to conduct a non-centralized rebuild procedure by managing a simultaneous and coordinated utilization of at least one operatable remote storage media that form the shared drive storage stack.
1. A data storage system, comprising:
(i) at least one target server that comprises a non-transitory storage media and configured to run an operating system designated to host data accessible over a data plane (DP) network;
(ii) at least one initiator server comprises a storage media and configured to run an operating system designated to access and expose at least one remote resource over the data plane (DP) network; and
(iii) at least one orchestrator configured to interact with each of said servers and designated to control a control plane (CP) of said DP network;
wherein the storage media of each of the at least one target server combined form a shared drive storage stack;
wherein the orchestrator is configured to utilize the CP in order to monitor and inspect operability and condition of the shared drive storage stack, and further configured to monitor, record and inspect proper operation of at least one block device stored within said shared drive storage stack;
wherein the orchestrator is configured to identify a malfunction affecting the shared drive storage stack; and
wherein the orchestrator is configured to conduct a non-centralized rebuild procedure by managing a simultaneous and coordinated utilization of at least one operatable, available and unexploited direct attached remote storage media that form the shared drive storage stack.
2. The system of claim 1 , wherein the shared drive storage stack comprises RAID components.
3. The system of claim 1 , wherein the orchestrator is configured to only use parts of at least one media storage within the storage system.
4. The system of claim 1 , wherein the orchestrator is configured to allocate new hot spare disk from a storage pool in case of failure.
5. The system of claim 1 , wherein the orchestrator is configured to instruct each server to add a selected spare storage media to each shared drive storage stack.
6. The system of claim 1 , wherein the orchestrator is configured to add a selected spare disk to the shared drive storage stack.
7. The system of claim 1 , wherein the orchestrator is configured to instruct each server to dismiss a failed disk in each shared drive storage stack and rebuilding new spare drive chunk with valid data.
8. The system of claim 1 , wherein connection between the servers is conducted using a multipath connection.
9. The system of claim 1 , wherein the orchestrator is configured to provide alerts and handle failures for remote block device connection between servers.
10. The system of claim 1 , wherein multiple initiators are configured to be created per network path to the initiator server.
11. The system of claim 1 , wherein the orchestrator is configured to identify a block device name on the remote server and on a local server, using information from the target server.
12. The system of claim 1 , wherein coordination between the initiator server and the target server is conducted using a CP orchestrator installed on each server.
13. The system of claim 1 , wherein communication between either a target or initiator server accessible via the DP network and the orchestrator is done using a designated software component installed on each of said servers.
14. The system of claim 1 , wherein the initiator server is configured to utilize a redundant array of independent disks (RAID) storage stack component (SSC) configured to provide data redundancy originated from multiple designated portions of the storage media at multiple target servers.
15. The system of claim 14 , wherein the RAID SSC is configured to provide data redundancy originated from combined multiple initiator paths originated from the designated portion of the storage media of at least two target servers.
16. The system of claim 15 , wherein the target servers are located at different locations and wherein the orchestration is allocated across different resiliency domains.
17. The system of claim 16 , wherein the orchestration allocated while considering different resiliency domains also considers maintaining system balance.
18. The system of claim 1 , wherein a designated portion of the storage media is allocated using a logical volume manager (LVM) SSC.
19. The system of claim 1 , wherein operations on each server may be implemented, wholly or partially, by a data processing unit (DPU).
20. A method for orchestrating and storing data, comprising steps of:
(i) enabling at least one target server that comprises a non-transitory storage media and configured to run an operating system to host data accessible over a data plane (DP) network;
(ii) enabling at least one initiator server configured to run an operating system to access and expose at least one remote resource over the data plane (DP) network; and
(iii) enabling at least one orchestrator configured to interact with each of said servers to control a control plane (CP) of said DP network;
wherein the storage media of each of the at least one target server combined form a shared drive storage stack;
wherein the orchestrator is configured to utilize the CP in order to monitor and inspect operability and condition of the shared drive storage stack, and further configured to monitor, record and inspect proper operation of at least one block device stored within said shared drive storage stack;
wherein the orchestrator is configured to identify a malfunction affecting the shared drive storage stack; and
wherein the orchestrator is configured to conduct a non-centralized rebuild procedure by managing a simultaneous and coordinated utilization of at least one operatable, available and unexploited direct attached remote storage media that form the shared drive storage stack.