Partial task messaging in a distributed storage system
A method for processing partial tasks in a distributed storage network (DSN) includes receiving a partial task request message for a DS execution unit, where the partial task request message includes corresponding partial tasks. The method continues by processing each partial task request message in accordance with the processing parameters to produce task request slice groupings, generating slices, such that each message is directed at a corresponding DS execution unit, and the sending the slice groupings and the task request slice groupings to the selected DS execution units for storage therein. The method continues by retrieving at least a decode threshold number of task response slices of one or more task response slice groupings from the DS execution units, decoding the task response slices, retrieving at least a decode threshold number of partial result slices, and decoding the partial results slices and processing the partial results to produce a result.
1. A method for execution by one or more processing modules of one or more computing devices of a distributed storage network (DSN), the method comprises:
receiving a slice grouping of contiguous data and a corresponding task request slice grouping;
retrieving the corresponding task request slice grouping to reproduce a partial task request message that includes at least one partial task and a portion of the contiguous data;
executing the partial task to produce a partial result;
generating a partial task response that includes a partial result status indicator based on the partial result;
processing the partial task response in accordance with processing parameters to produce a task response slice grouping;
facilitating storing the task response slice grouping as task response slices;
processing the partial result in accordance with the processing parameters to produce a partial result slice grouping; and
facilitating storing the partial result slice grouping as partial result slices.
2. The method of claim 1 , wherein the facilitating storing includes storing the slice grouping of contiguous data and a corresponding partial task request slice grouping in a local memory.
3. The method of claim 2 , wherein the local memory includes a mailbox.
4. The method of claim 3 , wherein the retrieving the corresponding task request slice grouping to reproduce a partial task request message includes retrieving a mail message from the mailbox.
5. The method of claim 1 , wherein the facilitating storing includes sending the partial result slice grouping to the DSN for storage therein.
6. The method of claim 5 , wherein the storage includes a return mailbox.
7. The method of claim 1 , wherein the partial result status indicator includes one of: a result ready status level or a result not ready status level.
8. The method of claim 1 further comprises utilizing zero information gain (ZIG) partial slice encoding to generate error coded task response slices based on one or more of: a number of participating pillars, the task response slice grouping, a previous task response slice grouping, an encoding matrix, an error coded data pillar number, or a zero information gain slice building approach.
9. The method of claim 1 , wherein the facilitate storing the task response slice grouping as the task response slices includes sending the task response slice grouping to the DSN for storage therein.
10. The method of claim 1 further comprises utilizing zero information gain (ZIG) partial slice encoding to generate error coded partial result slices based on one or more of: a number of participating pillars, the partial result slice grouping, a previous partial result slice grouping, an encoding matrix, an error coded data pillar number, or a zero information gain slice building approach.
11. A computing device of a group of computing devices of a distributed storage network (DSN), the computing device comprises:
an interface;
a local memory; and
a processing module operably coupled to the interface and the local memory, wherein the processing module functions to:
receive a slice grouping of contiguous data and a corresponding task request slice grouping;
retrieve the corresponding task request slice grouping to reproduce a partial task request message that includes at least one partial task and a portion of the contiguous data;
execute the partial task to produce a partial result;
generate a partial task response that includes a partial result status indicator based on the partial result;
process the partial task response in accordance with processing parameters to produce a task response slice grouping;
facilitate storing the task response slice grouping as task response slices;
process the partial result in accordance with the processing parameters to produce a partial result slice grouping; and
facilitate storing the partial result slice grouping as partial result slices.
12. The computing device of claim 11 , wherein the processing module stores the slice grouping of contiguous data and a corresponding partial task request slice grouping in a local memory.
13. The computing device of claim 12 , wherein the local memory includes a mailbox.
14. The computing device of claim 13 , wherein the retrieve the corresponding task request slice grouping to reproduce a partial task request message includes retrieving a mail message from the mailbox.
15. The computing device of claim 11 , wherein the partial result status indicator includes one of: a result ready status level or a result not ready status level.
16. The computing device of claim 11 further comprises utilizing zero information gain (ZIG) partial slice encoding to generate error coded task response slices based on one or more of: a number of participating pillars, the task response slice grouping, a previous task response slice grouping, an encoding matrix, an error coded data pillar number, or a zero information gain slice building approach.
17. The computing device of claim 11 , wherein the facilitate storing the task response slice grouping as the task response slices includes sending the task response slice grouping to the DSN for storage therein.
18. The computing device of claim 17 , wherein the storage includes a return mailbox.
19. The computing device of claim 11 further comprises utilizing zero information gain (ZIG) partial slice encoding to generate error coded partial result slices based on one or more of: a number of participating pillars, the partial result slice grouping, a previous partial result slice grouping, an encoding matrix, an error coded data pillar number, or a zero information gain slice building approach.
20. A method for execution by a storage unit of a distributed storage network (DSN), the method comprises:
receiving a slice grouping of contiguous data and a corresponding task request slice grouping;
retrieving the corresponding task request slice grouping to reproduce a partial task request message that includes at least one partial task and a portion of the contiguous data;
executing the partial task to produce a partial result;
generating a partial task response that includes a partial result status indicator based on the partial result;
processing the partial task response in accordance with processing parameters to produce a task response slice grouping;
facilitating storing the task response slice grouping as task response slices;
processing the partial result in accordance with the processing parameters to produce a partial result slice grouping; and
facilitating storing the partial result slice grouping as partial result slices.