Adaptive replication of dispersed data to improve data access performance
View Patent ↗Adaptive replication of data in a dispersed storage network (DSN) to improve data access performance. In various examples, a DSN storage unit determines that a frequency of slice access of an encoded data slice stored by the storage unit compares unfavorably to a first slice access threshold (e.g., a greater number of accesses than a threshold number of accesses over a given period of time). The storage unit then identifies at least one secondary storage unit and replicates the encoded data slice to generate a replicated encoded data slice. The replicated encoded data slice is then sent to the at least one secondary storage unit for storage therein. In addition, a slice storage location table is updated to associate the at least one secondary storage unit and the replicated encoded data slice such that future access requests for the encoded data slice may be re-directed to a secondary storage unit.
1. A method for execution by one or more processing modules of a dispersed storage network (DSN), the DSN including a plurality of storage units, the method comprises:
determining that a frequency of slice access of an encoded data slice stored in a first storage unit of the DSN exceeds a first slice access threshold, wherein a data segment of a data object is dispersed storage error encoded to produce a set of encoded data slices that includes the encoded data slice, and wherein a decode threshold number of encoded data slices of the set of encoded data slices is required to recover the data segment; and
in response to determining that the frequency of slice access of the encoded data slice exceeds the first slice access threshold:
identifying at least one secondary storage unit;
replicating the encoded data slice to generate a replicated encoded data slice;
sending the replicated encoded data slice to the at least one secondary storage unit for storage therein; and
updating a slice storage location table to associate the at least one secondary storage unit and the replicated encoded data slice.
2. The method of claim 1 further comprises:
determining that a frequency of slice access of the replicated encoded data slice is less than a second slice access threshold; and
updating the slice storage location table to disassociate the at least one secondary storage unit and the replicated encoded data slice.
3. The method of claim 1 , wherein determining at least one secondary storage unit is based on at least one of:
a current access performance level;
a performance requirement;
an estimated access performance level;
a request pattern;
a candidate secondary storage unit list;
a storage unit location;
a storage unit performance level; or
a storage unit Internet protocol (IP) address.
4. The method of claim 1 , wherein updating the slice storage location table to associate the at least one secondary storage unit and the replicated encoded data slice includes at least one of:
updating a slice storage location table maintained by the first storage unit;
updating a slice storage location table maintained by the at least one secondary storage unit; or
updating a slice storage location table maintained by a higher-level controller of the first storage unit.
5. The method of claim 1 further comprises:
prior to sending the replicated encoded data slice to the at least one secondary storage unit for storage therein, determining that the replicated encoded data slice is not stored in the at least one secondary storage unit.
6. The method of claim 1 further comprises:
obtaining the frequency of slice access of the encoded data slice based on at least one of:
a frequency of slice access query;
a lookup;
a list;
an error message;
a request; or
a command.
7. The method of claim 1 , wherein replicating the encoded data slice to generate a replicated encoded data slice includes rebuilding the encoded data slice utilizing at least the decode threshold number of other encoded data slices of the set of encoded data slices.
8. The method of claim 1 further comprises:
determining that an elapsed period of time since a last slice access of the replicated encoded data slice is less than a deletion time threshold;
in response to determining that the elapsed period of time since a last access of the replicated encoded data slice is less than the deletion time threshold, deleting the replicated encoded data slice from the at least one secondary storage unit; and
updating the slice storage location table to disassociate the at least one secondary storage unit and the replicated encoded data slice.
9. A computer readable storage medium comprises:
at least one memory section that stores operational instructions that, when executed by one or more processing modules of a dispersed storage network (DSN) that include a processor and a memory, causes the one or more processing modules to:
determine that a frequency of slice access of an encoded data slice stored in a first storage unit of the DSN exceeds a first slice access threshold, wherein a data segment of a data object is dispersed storage error encoded to produce a set of encoded data slices that includes the encoded data slice, and wherein a decode threshold number of encoded data slices of the set of encoded data slices is required to recover the data segment; and
in response to determining that the frequency of slice access of the encoded data slice exceeds the first slice access threshold:
identify at least one secondary storage unit;
replicate the encoded data slice to generate a replicated encoded data slice;
send the replicated encoded data slice to the at least one secondary storage unit for storage therein; and
update a slice storage location table to associate the at least one secondary storage unit and the replicated encoded data slice.
10. The computer readable storage medium of claim 9 further comprises:
the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to:
determine that a frequency of slice access of the replicated encoded data slice is less than a second slice access threshold; and
in response to determining that the frequency of slice access of the replicated encoded data slice is less than the second slice access threshold, update the slice storage location table to disassociate the at least one secondary storage unit and the replicated encoded data slice.
11. The computer readable storage medium of claim 9 further comprises:
the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to:
prior to sending the replicated encoded data slice to the at least one secondary storage unit for storage therein, determine that the replicated encoded data slice is not stored in the at least one secondary storage unit.
12. The computer readable storage medium of claim 9 , wherein identifying at least one secondary storage unit is based on at least one of:
a current access performance level;
a performance requirement;
an estimated access performance level;
a request pattern;
a candidate secondary storage unit list;
a storage unit location;
a storage unit performance level; or
a storage unit Internet protocol (IP) address.
13. The computer readable storage medium of claim 9 , wherein updating the slice storage location table to associate the at least one secondary storage unit and the replicated encoded data slice includes at least one of:
updating a slice storage location table maintained by the first storage unit;
updating a slice storage location table maintained by the at least one secondary storage unit; or
updating a slice storage location table maintained by a higher-level controller of the first storage unit.
14. The computer readable storage medium of claim 9 further comprises:
the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to:
obtain the frequency of slice access of the encoded data slice based on at least one of:
a frequency of slice access query;
a lookup;
a list;
an error message;
a request; or
a command.
15. A storage unit for use in a dispersed storage network (DSN) having a plurality of storage units, the storage unit comprises:
a network interface;
a memory; and
a processing module operably coupled to the network interface and the memory, wherein the processing module is configured to:
determine that a frequency of slice access of an encoded data slice stored in the memory exceeds a first slice access threshold, wherein a data segment of a data object is dispersed storage error encoded to produce a set of encoded data slices that includes the encoded data slice, and wherein a decode threshold number of encoded data slices of the set of encoded data slices is required to recover the data segment; and
in response to determining that the frequency of slice access of the encoded data slice exceeds the first slice access threshold:
identify at least one secondary storage unit;
replicate the encoded data slice to generate a replicated encoded data slice;
send, via the network interface, the replicated encoded data slice to the at least one secondary storage unit for storage therein; and
update a slice storage location table to associate the at least one secondary storage unit and the replicated encoded data slice.
16. The storage unit of claim 15 , wherein the processing module is further configured to:
determine that a frequency of slice access of the replicated encoded data slice is less than a second slice access threshold; and
in response to determining that the frequency of slice access of the replicated encoded data slice is less than the second slice access threshold, update the slice storage location table to disassociate the at least one secondary storage unit and the replicated encoded data slice.
17. The storage unit of claim 15 , wherein identifying at least one secondary storage unit is based on at least one of:
a current access performance level;
a performance requirement;
an estimated access performance level;
a request pattern;
a candidate secondary storage unit list;
a storage unit location;
a storage unit performance level; or
a storage unit Internet protocol (IP) address.
18. The storage unit of claim 15 , wherein updating the slice storage location table to associate the at least one secondary storage unit and the replicated encoded data slice includes at least one of:
updating a slice storage location table maintained in the memory of the storage unit;
updating a slice storage location table maintained by the at least one secondary storage unit; or
updating a slice storage location table maintained by a higher-level controller of the storage unit.
19. The storage unit of claim 15 , wherein the processing module is further configured to:
obtain the frequency of slice access of the encoded data slice based on at least one of:
a frequency of slice access query;
a lookup;
a list;
an error message;
a request; or
a command.
20. The storage unit of claim 15 , wherein replicating the encoded data slice to generate a replicated encoded data slice includes rebuilding the encoded data slice utilizing at least the decode threshold number of other encoded data slices of the set of encoded data slices.