ENCODED DATA SLICE CACHING IN A DISTRIBUTED STORAGE NETWORK
A distributed storage processing unit encodes data objects into multiple encoded data slices to prevent reconstruction of the original data object using a single encoded data slice, but to allow reconstruction using at least a threshold number of encoded data slices. The distributed storage processing unit can decide to whether and where to cache frequently requested data slices. When retrieving data slices related to a particular data object, a check can be made to determine if the data slices are cached in a temporary memory associated with the distributed storage processing unit, or elsewhere in the distributed storage network. This check can be facilitated by storing data slices and a hash table identifying the location of stored data slices in the same temporary memory.
1 . A method for use in a distributed storage processing unit, the method comprising:
receiving a request to retrieve requested information stored as encoded data slices in a distributed processing system, each encoded data slice including a data object encoded to prevent reconstruction of the data object using a single encoded data slice, and to permit reconstruction of the data object using at least a threshold number of encoded data slices;
determining whether relevant encoded data slices corresponding to at least part of the requested information are cached in temporary storage associated with a distributed storage processing unit;
if the relevant encoded data slices are cached in the temporary storage, retrieving the relevant encoded data slices from the temporary storage; and
if the relevant encoded data slices are not cached in the temporary storage, retrieving the relevant encoded data slices from a dispersed storage network unit.
2 . The method of claim 1 , further comprising:
if the relevant encoded data slices are not cached in the temporary storage, determining whether to cache the relevant encoded data slices in the temporary storage; and
if the determining indicates that the encoded data slices are to be cached:
storing the relevant encoded data slices in the temporary storage; and
storing names of the encoded data slices in a hash table.
3 . The method of claim 2 , further comprising:
determining whether to cache the relevant encoded data slices in the temporary storage based on access tracking information associated with the relevant encoded data slices.
4 . The method of claim 3 , wherein the access tracking information, on which the decision to cache is based, is selected from a group consisting essentially of:
a frequency of accessing data slices associated with the information, network loading levels, a priority level, and a security level.
5 . The method of claim 2 , further comprising:
limiting storage of the relevant encoded data slices in the temporary storage to a read threshold number of encoded data slices per data object.
6 . The method of claim 1 , further comprising:
de-slicing and decoding the relevant encoded data slices to produce the data object; and
sending the data object to a requestor.
7 . The method of claim 1 , further comprising:
verifying that at least a read threshold number of relevant encoded data slices that have been retrieved are valid.
8 . The method of claim 1 , further comprising:
removing data slices from the temporary storage in response to a frequency of access dropping below a threshold value.
9 . The method of claim 1 , further comprising:
removing data slices from the temporary storage in response to at least one of a security level of the requested information, a priority level of the requested information, a network loading level, and a memory loading level.
10 . A distributed storage processing unit comprising:
an interface to receive a request to retrieve a data object stored as a plurality of data slices in a distributed processing system, each of the plurality of data slices including an instance of the data object encoded to prevent reconstruction of the data object using a single data slice, and to permit reconstruction of the data object using at least a threshold number of data slices;
a processor to:
determine whether data slices corresponding to the data object are cached in a temporary storage;
retrieve the data slices from the temporary storage if the data slices are cached in the temporary storage; and
retrieve the data slices from a dispersed storage network unit if the data slices are not cached in the temporary storage.
11 . The distributed storage processing unit of claim 10 , the processor further to:
determine whether to cache the data slices in the temporary storage if the data slices are not cached in the temporary storage; and
if the processor determines that the encoded data slices are to be cached:
store the relevant encoded data slices in the temporary storage; and
store names of the encoded data slices in a hash table.
12 . The distributed storage processing unit of claim 11 , the processor further to:
determine whether to cache the data slices based on access tracking information associated with the data slices.
13 . The distributed storage processing unit of claim 12 , wherein the access tracking information used by the processor determine whether to cache the data slices is selected from a group consisting essentially of: a frequency of accessing data slices associated with the information, network loading levels, a priority level, and a security level.
14 . The distributed storage processing unit of claim 10 , further comprising:
a grid module to de-slice and decode the data slices to produce the data object; and
an interface to send the data object to a requestor.
15 . The distributed storage processing unit of claim 10 , the processor further to:
verify a validity of the data slices.
16 . The distributed storage processing unit of claim 10 , further comprising:
a temporary storage including a slice memory and a hash table.
17 . A method comprising:
tracking a retrieval frequency of encoded data slices associated with a requested data object, each encoded data slice including an instance of the data object encoded to prevent reconstruction of the data object using a single encoded data slice, and to permit reconstruction of the data object using at least a read threshold number of data slices;
caching at least the read threshold number of encoded data slices in a local storage based, at least in part, on the retrieval frequency comparing favorably to a frequency threshold;
in response to caching an encoded data slice, updating a hash table stored in the local storage;
accessing the hash table in response to receiving a request for the requested data object, to determine if encoded data slices associated with the requested data object are cached in the local storage; and
retrieving the encoded data slices from the local storage if the hash table indicates that the encoded data slices are cached.
18 . The method of claim 17 , further comprising:
retrieving the encoded data slices from a memory associated with a dispersed storage network unit if the hash table indicates that the encoded data slices are not cached locally.
19 . The method of claim 17 , further comprising:
caching at least the read threshold number of encoded data slices in the local storage based, at least in part, on access tracking information.
20 . The method of claim 19 , wherein the access tracking information is selected from a group consisting essentially of: network loading levels, a priority level, and a security level.
21 . The method of claim 17 , further comprising:
de-slicing and decoding the relevant encoded data slices to produce the requested data object; and
sending the data object to a requestor.
22 . The method of claim 17 , further comprising:
removing data slices from the local storage in response to a frequency of access dropping below a threshold value.