Resilient distributed storage system

1. One or more computing systems for providing distributed storage of data, the one or more computing systems comprising:

one or more computer-readable storage mediums for storing computer-executable instructions of:

endpoint nodes, gateway nodes, and one or more gateway name servers, wherein:

the one or more gateway name servers receive resolution requests from clients to resolve to a node address of one of said gateway nodes;

the gateway nodes form a mesh of gateway nodes that receive retrieval requests from clients to access one of said endpoint nodes and redirect the retrieval requests to said endpoint nodes, each retrieval request being received by said one of said gateway nodes whose node address was resolved to by the one or more gateway name servers;

the endpoint nodes:

receive a data retrieval request from a client based on being redirected thereto by said one of said gateway nodes;

upon receipt of the data retrieval request, sends the data to the client if a determination is made that the data is stored locally; and

upon receipt of the data retrieval request, forwards the retrieval request to another endpoint node if a determination is made that the data is not stored locally; and

one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage mediums.

2. The one or more computing systems of claim 1 wherein a data retrieval request identifies data by a hash of the data.

3. The one or more computing systems of claim 1 wherein when an endpoint node that forwards a data retrieval request, the endpoint node receives the data from the other endpoint node and sends the data to the client.

4. The one or more computing systems of claim 1 wherein a gateway node identifies a ranking of endpoint nodes based on accessibility and redirects accesses based on the ranking.

5. The one or more computing systems of claim 4 wherein the accesses are redirected in a round-robin manner among highest-ranking endpoint nodes.

6. The one or more computing systems of claim 1 wherein a gateway node receives from a client a store request to store data and requests a subset of the endpoint nodes to store data.

7. The one or more computing systems of claim 1 wherein a gateway node received a delete request to delete data and requests each endpoint node to delete the data.

8. The one or more computing systems of claim 1 wherein a console node enforces permissions of clients to store and delete data.

9. The one or more computing systems of claim 1 wherein the gateway name servers resolve to gateway nodes in a round-robin manner.

10. The one or more computing systems of claim 1 wherein the computer-executable instructions further include instructions of a navigation directory wherein the navigation directory maps a client data identifier to an endpoint node data identifier that is used by an endpoint node to identify data.

11. The one or more computing systems of claim 1 wherein the gateway nodes form a self-healing mesh.

12. A method performed by one or more computing systems of one or more gateway nodes that provide access to endpoint nodes that provide access to files stored by endpoint nodes, the method comprising:

when a request is received to store a file:

generating a hash of the file;

selecting a subset of said endpoint nodes based on a round-robin selection of endpoint nodes that are highly ranked based on their accessibility, wherein ranking of said endpoint nodes is determined by consensus of said one or more gateway nodes; and

sending to the selected subset of said endpoint nodes a request to store the file along with a hash of the file; and

when a request is received to delete a file identified by a hash,

sending to each of said endpoint nodes a request to delete the file identified by the hash; and

when a request is received to redirect an access to a respective endpoint node:

selecting the respective endpoint node; and

redirecting the access to the respective endpoint node.

13. A method performed by one or more computing systems of an endpoint node that stores files of a distributed storage system, the method comprising:

receiving retrieval requests to retrieve at least one file identified by at least one hash, each retrieval request being redirected to the endpoint node by a gateway node, the gateway node being identified by a gateway domain name system;

determining whether the at least one file is stored at the endpoint node based on said at least one hash of the at least one file;

when a file is stored at the endpoint node, providing the file in response to the retrieve request; and

when a file is not stored at the endpoint node;

forwarding the request to another endpoint node;

receiving the file from the another endpoint node;

storing the file at the endpoint node; and

providing the file in response to the retrieve request.

14. The method of claim 13 further comprising receiving from the gateway node a request to store the at least one file and storing the at least one file locally.

15. The method of claim 14 further comprising receiving from a gateway node a request to delete the at least one file and deleting a local copy of the at least one file.

16. The method of claim 13 wherein the at least one file has a retention flag and further comprising deleting local copies of the at least one file whose retention flag indicates that the at least one file need not be retained.

17. The method of claim 13 wherein the gateway node redirects to the endpoint node based on a ranking of accessibility of the endpoint nodes.