Two-level structured overlay design for cluster management in a peer-to-peer network
A method and system for designing file replication schemes in file sharing systems consider node storage constraints and node up/down statistics, file storage costs, and file transfer costs among the nodes, user request rates for the files, and user specified file availability requirements. Based on these considerations, a systematic method for designing file replication schemes can be implemented. The method first determines the number of copies of the files to be stored in the system to achieve the desired goal (e.g., to satisfy file availability requirements, or to maximize the system hit rate), and then selects the nodes at which to store the file copies to minimize the total expected cost. The file replication scheme for a peer-to-peer file sharing system in a distributed and adaptive manner can scale to a large number of nodes and files and can handle changes in the user request pattern over time.
1. A method for determining storage locations for a resource and copies thereof, a storage location being associated with a node in a network of nodes, the network of nodes forming a system, comprising:
(a) determining, when a resource first enters the system for storage thereof, a number of copies of the resource to be stored, based upon a probability that a node will be down, the copies of the resource to be stored being determined copies;
(b) randomly selecting, when the resource first enters the system for storage thereof, a number of nodes to store the determined copies thereon, the number of nodes being equal to the number of determined copies;
(c) storing the determined copies on the randomly selected nodes, the determined copies stored on the randomly selected nodes being stored determined copies;
(d) reselecting, when a predetermined number of requests have been received for the stored resource, nodes to store the stored determined copies thereon, based upon a minimization of a storage cost; and
(e) storing the stored determined copies on the reselected nodes, the stored determined copies being the determined copies previously stored on the randomly selected nodes.
2. The method as claimed in claim 1 , wherein the storage cost includes a capacity of nodes to store the resources.
3. The method as claimed in claim 1 , wherein said reselecting the nodes to store the stored determined copies thereon is based upon a minimization of a storage cost and a minimization of a transfer cost.
4. The method as claimed in claim 2 , wherein said reselecting the nodes to store the stored determined copies thereon is based upon a minimization of a storage cost and a minimization of a transfer cost.
5. The method as claimed in claim 1 , wherein said determining the number of copies is based upon calculating the sufficient number of copies to meet a predefined probability that a request for a resource can be satisfied.
6. The method as claimed in claim 1 , wherein said determining the number of copies is based upon maximizing a hit rate of the system.