Electronic storage system
Methods and systems for electronic storage are provided. A storage system comprises a plurality of storage system front ends, a plurality of storage system back ends, and a plurality of solid state drive (SSD) agents. Each storage system front end resides on a server of a plurality of servers. Each server of the plurality of servers comprises one or more storage system back ends of the plurality of storage system back ends. Each storage system front end is able to receive I/O requests and relay information associated with the I/O requests to a relevant storage system back end. The relevant storage system back end communicates metadata associated with the I/O request to an SSD via an SSD agent.
1. A method, comprising: receiving metadata, associated with an I/O request, via a storage system back end; communicating the I/O request metadata to a solid state drive (SSD) via an SSD agent; migrating the I/O request metadata from the SSD to an object store as a background asynchronous process, wherein: the I/O request is a write operation, the SSD is one of a plurality of SSDs in a storage system, the storage system back end is one of a plurality of storage system back ends, the plurality of storage system back ends are organized into a plurality of erasure-coded stripes, each erasure-coded stripe spans more than one failure domain, and each of a plurality of servers comprises one or more storage system back ends of the plurality of storage system back ends.
2. The method of claim 1 , wherein the object store is cloud-based.
3. The method of claim 1 , comprising: adding a new SSDs to the plurality of SSDs; and redistributing data already written to the plurality of SSDs, wherein the redistribution is a virtualization of the storage system across the plurality of SSDs.
4. The method of claim 1 , comprising: detecting a failure of the SSD; rebuilding data of the SSD via data distributed across all SSDs of the plurality of SSDs that have not failed; and load balancing the data distributed across all SSDs of the plurality of SSDs that have not failed.
5. The method of claim 4 , wherein rebuilding begins after the failure has been detected for a specified timeout.
6. The method of claim 4 , comprising: detecting a resolution of the failure of the SSD; and aborting the rebuilding according to the resolution.
7. The method of claim 1 , wherein the method is caused to be performed by a machine that is executing code that is stored on a non-transitory machine-readable storage.
8. A storage system comprising:
a plurality of storage system back ends organized into a plurality of erasure-coded stripes, wherein each erasure-coded stripe spans more than one failure domain;
a plurality of servers, wherein each of the servers is associated with one or more storage system back ends of the plurality of storage system back ends;
a plurality of solid state drives (SSDs); and
a plurality of SSD agents, wherein when metadata associated with an I/O request is relayed to a particular storage system back end of the plurality of storage system back ends:
the I/O request metadata is communicated, via an SSD agent, between a storage system back end, of the plurality of storage system back ends, and an SSD, of the plurality of SSDs;
the I/O request metadata is written to the SSD; and
the I/O request metadata is migrated from the SSD to an object store as a background asynchronous process.
9. The storage system of claim 8 , wherein the object store is cloud-based.
10. The storage system of claim 8 , comprising:
a new SSD, wherein:
the new SSD is added to the plurality of SSDs,
data already written to the plurality of SSDs is redistributed, and
the redistribution is a virtualization of the storage system across the plurality of SSDs comprising the new SSD.
11. The storage system of claim 8 , wherein:
the detection of a failure of the SSD causes data of the SSD to be rebuilt via data that is distributed across all SSDs of the plurality of SSDs that have not failed, and
the distributed data is load balanced across all SSDs of the plurality of SSDs that have not failed.
12. The storage system of claim 11 , wherein the rebuilding begins after the failure has been detected for a specified timeout.
13. The storage system of claim 11 , wherein the rebuilding is aborted if the failure of the SSD is resolved before the rebuilding is complete.
14. A method caused to be performed by a machine that is executing code stored on a non-transitory machine-readable storage, the method comprising:
configuring each of a plurality of servers with one or more storage system back ends of a plurality of storage system back ends;
organizing the plurality of storage system back ends into a plurality of erasure-coded stripes, wherein each erasure-coded stripe spans more than one failure domain;
relaying metadata associated with an I/O request to a storage system back end of the plurality of storage system back ends;
communicating the I/O request metadata, via a solid state drive (SSD) agent of a plurality of SSD agents, between the storage system back end and an SSD of a plurality of SSDs;
writing the I/O request metadata to the SSD; and
migrating the I/O request metadata from the SSD to an object store as a background asynchronous process.
15. The method of claim 14 , wherein the object store is cloud-based.
16. The method of claim 14 , comprising:
adding a new SSDs to the plurality of SSDs of the storage system; and
redistributing data already written to the plurality of SSDs, wherein the redistribution is a virtualization of the storage system across the plurality of SSDs.
17. The method of claim 14 , comprising:
detecting a failure of the SSD;
rebuilding data of the SSD via data distributed across all SSDs of the plurality of SSDs that have not failed; and
load balancing the data distributed across all SSDs of the plurality of SSDs that have not failed.
18. The method of claim 17 , wherein rebuilding begins after the failure has been detected for a specified timeout.
19. The method of claim 17 , comprising:
detecting a resolution of the failure of the SSD; and
aborting the rebuilding according to the resolution.
20. The method of claim 14 , wherein the I/O request is a write operation.