Peer-to-peer redundant file server system and methods
View Patent ↗Peer-to-peer redundant file server system and methods include clients that determine a target storage provider to contact for a particular storage transaction based on a pathname provided by the filesystem and a predetermined scheme such as a hash function applied to a portion of the pathname. Servers use the same scheme to determine where to store relevant file information so that the clients can locate the file information. The target storage provider may store the file itself and/or may store metadata that identifies one or more other storage providers where the file is stored. A file may be replicated in multiple storage providers, and the metadata may include a list of storage providers from which the clients can select (e.g., randomly) in order to access the file.
1. A method of distributing processing load across a plurality of storage providers, the method comprising:
determining that multiple clients desire access to a file stored by a given storage provider;
replicating the file in at least one additional storage provider, such that the given storage provider and at least one additional storage provider each store an instantiation of the file; and
allowing clients to access any of the instantiations of the file so as to distribute processing load across the storage providers, wherein allowing clients to access an of the instantiations of the file comprises, for a client of the multiple clients:
identifying a target storage provider of the plurality of storage providers that stores metadata corresponding to the file;
accessing the identified target storage provider to retrieve a list containing references to storage providers of the plurality of storage providers that store an instantiation of the file; and
selecting, from the retrieved list, at least one storage provider from which to access the file.
2. A method according to claim 1 , wherein allowing clients to access any of the instantiations of the file comprises:
specifying a different one of the storage providers for each of the clients.
3. A method according to claim 1 , wherein the target storage provider is selected based at least in part on a hash performed on at least a portion of a path associated with the file.
4. A method according to claim 1 , wherein the list contains references to at least two storage providers storing at least a first instantiation of the file and a second instantiation of the file, wherein the first instantiation of the file is a different version from the second instantiation of the file.
5. A method according to claim 4 , wherein the list further contains version numbers for each of the stored instantiations of the file.
6. A method according to claim 1 , wherein the client randomly selects a storage provider from the retrieved list from which to access the file.
7. A method according to claim 1 , wherein the list contains a reference to the target storage provider, and wherein the target storage provider stores an instantiation of the file.
8. A storage system comprising:
a plurality of clients;
a storage environment in communication with the plurality of clients over a communication network, wherein the storage environment comprises a plurality of storage providers, and is configured to:
determine that multiple clients of the plurality of clients desire access to a file stored by a given storage provider;
replicate the file in at least one additional storage provider of the plurality of storage providers, such that the given storage provider and at least one additional storage provider each store an instantiation of the file; and
allow the clients to access any of the instantiations of the file so as to distribute processing load across the storage providers, wherein a client of the multiple clients accessing an instantiation of the file by:
identifying a target storage provider of the plurality of storage providers that stores metadata corresponding to the file;
accessing the identified target storage provider to retrieve a list containing references to storage providers of the plurality of storage providers that store an instantiation of the file; and
selecting, from the retrieved list, at least one storage provider from which to access the file.
9. A storage system according to claim 8 , wherein a different one of the storage providers is specified for each of the multiple clients for accessing a instantiation of the file.
10. A storage system according to claim 8 , wherein the target storage provider is selected based at least in part on a hash performed on at least a portion of a path associated with the file.
11. A storage system according to claim 8 , wherein the list contains references to at least two storage providers storing at least a first instantiation of the file and a second instantiation of the file, wherein the first instantiation of the file is a different version from the second instantiation of the file.
12. A storage system according to claim 11 , wherein the list further contains version numbers for each of the stored instantiations of the file.
13. A storage system according to claim 8 , wherein the client randomly selects a storage provider from the retrieved list from which to access the file.
14. A storage system according to claim 8 , wherein the list contains a reference to the target storage provider, and wherein the target storage provider stores an instantiation of the file.