Pre-fetching data segments stored in a dispersed storage network
View Patent ↗A method begins by a processing module receiving a data segment retrieval request regarding a data segment, which is stored in a dispersed storage network (DSN) memory. The method continues with the processing module processing the data segment retrieval request, determining pre-fetch segment buffering information, and when the pre-fetch segment buffering information indicates pre-fetching one or more other data segments, generating one or more pre-fetch segment retrieval requests for the one or more other data segments, receiving, one or more sets of at least a decode threshold number of encoded data slices, decoding, in accordance with a dispersed storage error coding function, the one or more sets of at least a decode threshold number of encoded data slices to reproduce the one or more other data segments, and updating a pre-fetch segment buffer with the one or more other data segments.
1. A method for execution by a computing device, the method comprises:
determining to pre-fetch a number of encoded data segments from dispersed storage network (DSN) memory, wherein a data segment of data is disperse storage error encoded to produce one of the number of encoded data segments to include a set of encoded data slices and wherein the number of encoded data segments includes a number of sets of encoded data slices;
for the number of encoded data segments:
determining an encoded data slice retrieval request pattern for requesting retrieval of encoded data slices of the number of sets of encoded data slices from storage units of the DSN memory based on one or more factors of a set of factors, wherein the set of factors includes data consumption, response latency information, processing consumption, network traffic, response history, and reliability information;
sending, to a first storage unit of the storage units, encoded data slice requests in accordance with a first portion of the encoded data slice retrieval request pattern;
sending, to a second storage unit of the storage units, encoded data slice requests in accordance with a second portion of the encoded data slice retrieval request pattern;
receiving, for an encoded data segment of the number of encoded data segments, a threshold number of encoded data slices prior to a specific retrieval request for the encoded data segment; and
storing the threshold number of encoded data slices in a pre-fetch buffer.
2. The method of claim 1 , wherein the determining the encoded data slice retrieval request pattern further comprises:
identifying the storage units that are storing encoded data slices of the number of sets of encoded data slices.
3. The method of claim 1 further comprises:
receiving the specific retrieval request for the encoded data segment;
determining that a decode threshold number of encoded data slices are stored in the pre-fetch buffer;
retrieving the decode threshold number of encoded data slices from the pre-fetch buffer; and
decoding the decode threshold number of encoded data slices to produce a decoded data segment.
4. The method of claim 1 further comprises:
sending, to each storage unit of the storage units, encoded data slice requests in accordance with a corresponding portion of the encoded data slice retrieval request pattern; and
receiving, for the encoded data segments, the threshold number of encoded data slices of a particular one of the encoded data segments prior to a retrieval request of a sequence of retrieval requests for the encoded data segments, wherein the retrieval request is a request for the particular one of the encoded data segments.
5. The method of claim 1 further comprises:
determining a time interval for receiving the threshold number of encoded data slices for the number of encoded data slices;
sending, to the first storage unit of the storage units, a first number of the encoded data slice requests during a first portion of the time interval and a second number of the encoded data slice requests during a second portion of the time interval; and
sending, to the second storage unit of the storage units, the encoded data slice requests during the first portion of the time interval.
6. The method of claim 1 further comprises:
determining the number of encoded data segments based on a data segment retrieval trend.
7. A non-transitory computer readable storage device comprises:
a first memory section that stores operational instructions that, when executed by a computing device, causes the computing device to:
determine to pre-fetch a number of encoded data segments from dispersed storage network (DSN) memory, wherein a data segment of data is disperse storage error encoded to produce one of the number of encoded data segments to include a set of encoded data slices and wherein the number of encoded data segments includes a number of sets of encoded data slices;
for the number of encoded data segments:
a second memory section that stores operational instructions that, when executed by the computing device, causes the computing device to:
determine an encoded data slice retrieval request pattern for requesting retrieval of encoded data slices of the number of sets of encoded data slices from storage units of the DSN memory based on one or more factors of a set of factors, wherein the set of factors includes data consumption, response latency information, processing consumption, network traffic, response history, and reliability information;
a third memory section that stores operational instructions that, when executed by the computing device, causes the computing device to:
send, to a first storage unit of the storage units, encoded data slice requests in accordance with a first portion of the encoded data slice retrieval request pattern;
send, to a second storage unit of the storage units, encoded data slice requests in accordance with a second portion of the encoded data slice retrieval request pattern;
receive, for an encoded data segment of the number of encoded data segments, a threshold number of encoded data slices prior to a specific retrieval request for the encoded data segment; and
a fourth memory section that stores operational instructions that, when executed by the computing device, causes the computing device to:
store the threshold number of encoded data slices in a pre-fetch buffer.
8. The non-transitory computer readable storage device of claim 7 , wherein the second memory section further stores operational instructions that, when executed by the computing device, causes the computing device to determine the encoded data slice retrieval request pattern by:
identifying the storage units that are storing encoded data slices of the number of sets of encoded data slices.
9. The non-transitory computer readable storage device of claim 7 further comprises:
a fifth memory section that stores operational instructions that, when executed by the computing device, causes the computing device to:
receive the specific retrieval request for the encoded data segment;
determine that a decode threshold number of encoded data slices are stored in the pre-fetch buffer;
retrieve the decode threshold number of encoded data slices from the pre-fetch buffer; and
decode the decode threshold number of encoded data slices to produce a decoded data segment.
10. The non-transitory computer readable storage device of claim 7 , wherein the third memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:
send, to each storage unit of the storage units, encoded data slice requests in accordance with a corresponding portion of the encoded data slice retrieval request pattern; and
receive, for the encoded data segments, the threshold number of encoded data slices of a particular one of the encoded data segments prior to a retrieval request of a sequence of retrieval requests for the encoded data segments, wherein the retrieval request is a request for the particular one of the encoded data segments.
11. The non-transitory computer readable storage device of claim 7 , wherein the third memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:
determine a time interval for receiving the threshold number of encoded data slices for the number of encoded data slices;
send, to the first storage unit of the storage units, a first number of the encoded data slice requests during a first portion of the time interval and a second number of the encoded data slice requests during a second portion of the time interval; and
send, to the second storage unit of the storage units, the encoded data slice requests during the first portion of the time interval.
12. The non-transitory computer readable storage device of claim 7 , wherein the first memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:
determine the number of encoded data segments based on a data segment retrieval trend.