System and method of implementing an object storage infrastructure for cloud-based services

1. An object storage system comprising:

a plurality of storage nodes, each of said storage nodes including memory for storing digital objects associated with clients;

at least one client application server being operative to

establish a network connection with a client over an inter-network,

receive an indication of an object to be uploaded by said client,

generate an upload request associated with said object,

receive said object from said client via said inter-network; and

at least one cloud application server being operative to

receive said upload request from said client application server,

select at least two storage nodes from said plurality of storage nodes,

receive said object from said client application server, and

cause said object to be stored on each of said selected storage nodes;

a storage node recovery module operative to rebuild one of said plurality of storage nodes, including said digital objects stored on said one of said plurality of storage nodes, when said one of said storage nodes fails; and

an object database storing a plurality of object records associated with a plurality of stored digital objects, each said object record indicating multiple ones of said storage nodes storing an associated digit object.

2. The system of claim 1 , wherein each of said plurality of storage nodes exposes a HyperText Transfer Protocol (HTTP) interface.

3. The system of claim 1 , wherein the number of said storage nodes that said cloud application server is operative to select is configurable.

4. The system of claim 1 , wherein:

said cloud application server is further operative to cause an object record associating said object and said selected storage nodes to be created in said object database.

5. The system of claim 4 , wherein said object database is sharded into a plurality of shards such that each shard includes some of said plurality of object records.

6. The system of claim 5 , wherein:

said cloud application server is further operative to generate a Universally Unique Identifier (UUID) associated with said object;

said UUID includes a first field containing data identifying one of said plurality of shards containing said object record associated with said object; and

said UUID includes a second field containing an object identifier identifying said object record among said plurality of object records.

7. The system of claim 4 , further comprising:

at least one cloud object server; and wherein

said cloud object server is operative to

receive a download request for said object,

query said cloud application server for storage node information facilitating the retrieval of said object from any of said selected storage nodes,

request said object from any one of said selected storage nodes identified by said storage node information, and

provide said object to said client application server.

8. The system of claim 7 , wherein:

said object database is sharded into a plurality of shards;

each shard includes some of said plurality of object records; and

said download request includes a Universally Unique Identifier (UUID) comprising a shard identifier identifying one of said plurality of shards containing said object record.

9. The system of claim 8 , wherein:

said cloud object server is operative to provide said UUID to said cloud application server as part of said request for said storage node information; and

said cloud application server is further operative to use said UUID to query said object database to return said storage node information.

10. The system of claim 4 , wherein said cloud application server is further operative to:

receive an indication that said object is to be deleted; and

modify said object record to indicate that said object is marked for deletion.

11. The system of claim 10 , wherein said cloud application server is further operative to cause said object to be deleted from each of said selected storage nodes.

12. The system of claim 1 , wherein said cloud application server is further operative to generate a Universally Unique Identifier (UUID) associated with said object.

13. The system of claim 12 , further comprising:

a file system database storing information defining a virtual file system associated with said client and accessible to said client application server; and wherein

said UUID is stored in said file system database.

14. The system of claim 13 , further comprising:

at least one cloud object server; and wherein

said client application server is further operative to

receive a download command for said object from said client,

retrieve said UUID from said file system database, and

provide said UUID to said cloud object server; and

said cloud object server is operative to use said UUID to obtain information facilitating the retrieval of said object from any of said selected storage nodes.

15. The system of claim 1 , further comprising:

a file system database storing data defining a virtual file system associated with said client and being accessible to said client application server; and wherein

said client application server is further operative to store a virtual object record associated with said object in said file system database.

16. The system of claim 15 , wherein said client application server is further operative to:

receive a delete command from said client requesting that said object be deleted; and

modify said virtual object record to indicate that said object is to be deleted.

17. The system of claim 15 , wherein:

said file system database is sharded into a plurality of shards; and

each of said plurality of shards includes data associated with only one of said clients.

18. The system of claim 1 , further comprising:

a plurality of cloud object servers each being operative to retrieve said object from one of said selected storage nodes and serve said object to said client application server responsive to receiving a download request for said object;

a plurality of said cloud application servers; and

a load balancer operative to

receive said upload request from said client application server and provide it to a selected one of said plurality of cloud application servers and

receive said download request from said client application server and provide it to a selected one of said plurality of cloud object servers.

19. The system of claim 1 , wherein:

said storage node recovery module is operative to rebuild said failed storage node as a rebuilt storage node by using said plurality of object records to identify ones of said digital objects stored on said failed storage node and copy said identified digital objects from other ones of said plurality of storage nodes to said rebuilt storage node.

20. The system of claim 1 , wherein:

said plurality of object records are sharded by storage node; and

said storage node recovery module is operative to rebuild said failed storage node as a rebuilt storage node by identifying a shard associated with said failed storage node and copying digital objects associated with the object records contained in said identified shard from other ones of said plurality of storage nodes to said rebuilt storage node.

21. The system of claim 1 , further comprising a rebalance module operative to move digital objects stored on said one of said plurality of storage nodes to other ones of said plurality of storage nodes.

22. The system of claim 21 , wherein:

said rebalance module is operative to modify at least some of said plurality of object records.

23. The system of claim 1 , further comprising an object auditor operative to determine whether said object stored on one or more of said selected storage nodes has degraded.

24. The system of claim 1 , wherein:

said client application server is further operative to receive a second object from said client via said inter-network; and

said cloud application server is further operative to

select at least two storage nodes from said plurality of storage nodes for said second object that are different from said at least two storage nodes selected for said object and

cause said second object to be stored on each of said storage nodes selected for said second object.

25. The system of claim 1 , wherein multiple ones of said plurality of storage nodes are selected on an object-by-object basis for additional objects uploaded by said client.

26. An object storage system comprising:

a plurality of storage nodes, each of said storage nodes including memory for storing digital objects associated with clients;

at least one client application server being operative to

establish a network connection with a client over an inter-network,

receive an indication of an object to be uploaded by said client,

generate an upload request associated with said object, and

receive said object from said client via said inter-network;

means for selecting at least two storage nodes from said plurality of storage nodes in response to said upload request;

means for streaming said object from said client application server to each of said selected storage nodes such that said object is stored on each of said selected storage nodes;

a storage node recovery module operative to rebuild one of said plurality of storage nodes, including said digital objects stored on said one of said plurality of storage nodes, when said one of said storage nodes fails; and

an object database storing a plurality of object records associated with a plurality of stored digital objects, each said object record indicating multiple ones of said storage nodes storing an associated digit object.

27. The system of claim 26 , wherein:

said storage node recovery module is operative to rebuild said failed storage node as a rebuilt storage node by using said plurality of object records to identify ones of said digital objects stored on said failed storage node and copy said identified digital objects from other ones of said plurality of storage nodes to said rebuilt storage node.