DYNAMICALLY SHIFTING TASKS IN DISTRIBUTED COMPUTING DATA STORAGE
A method for dynamically shifting data-related tasks in a dispersed storage network (DSN). In an embodiment, a first dispersed storage and task (DST) execution unit (or computing device) of the DSN determines an incremental partial task execution capacity level, which is compared to a threshold level. When the partial task execution capacity level is above the threshold, the first DST execution unit selects one or more locally-stored encoded data slices which are also stored in a second DST execution unit. The first DST execution unit further obtains, from the second DST execution unit, at least one partial task relating to the encoded data slices. The first DST execution unit subsequently performs the at least one partial task on the one or more encoded data slices to produce partial results for use by the second DST execution unit or a device associated with assignment of the at least one partial task.
1 . A method for execution by a first dispersed storage and task (DST) execution unit of a set of DST execution units, the method comprises:
determining an incremental partial task execution capacity level of the first DST execution unit;
comparing the incremental partial task execution capacity level of the first DST execution unit to a threshold level;
in response to determining that the incremental partial task execution capacity level of the first DST execution unit is above the threshold level, selecting one or more encoded data slices of a slice group stored by the first DST execution unit, wherein the one or more encoded data slices are additionally stored by a second DST execution unit of the set of DST execution units;
obtaining, from the second DST execution unit, at least one partial task associated with the one or more encoded data slices of the slice group; and
facilitating execution of the at least one partial task on the one or more encoded data slices of the slice group stored by the first DST execution unit to produce partial results.
2 . The method of claim 1 further comprises:
sending the partial results to at least one of the second DST execution unit or a device associated with assignment of the at least one partial task.
3 . The method of claim 1 , wherein obtaining the at least one partial task associated with the one or more encoded data slices of the slice group includes sending a partial task retrieval request to the second DST execution unit.
4 . The method of claim 1 , wherein obtaining the at least one partial task associated with the one or more encoded data slices of the slice group includes:
retrieving DST allocation information relating to the at least one partial task and data partition information regarding the one or more encoded data slices; and
updating the DST allocation information to indicate that the second DST execution unit is no longer affiliated with execution of the at least one partial task.
5 . The method of claim 1 , wherein facilitating execution of the at least one partial task includes:
retrieving the one or more encoded data slices from a memory of the first DST execution unit; and
executing the at least one partial task on the retrieved one or more encoded data slices.
6 . The method of claim 1 , wherein facilitating execution of the at least one partial task includes updating a local task queue of the first DST execution unit to include the at least one partial task.
7 . The method of claim 1 , wherein selecting the one or more encoded data slices of the slice group stored by the first DST execution unit includes:
obtaining a partial task to slice identifier (ID) list; and
based on the partial task to slice ID list, identifying the one or more encoded data slices of the slice group.
8 . The method of claim 1 , wherein selecting the one or more encoded data slices of the slice group stored by the first DST execution unit includes generating and sending a partial task execution request requesting to execute at least one partial task relating to one or more encoded data slices that are stored in the first DST execution unit.
9 . The method of claim 1 , wherein the at least one partial task includes an unexecuted partial task assigned to the second DST execution unit.
10 . A first computing device for use in a dispersed storage network (DSN), the first computing device comprises:
a network interface;
a local memory; and
a processing module operably coupled to the local memory and the network interface, the processing module configured to:
determine an incremental partial task execution capacity level of the first computing device;
compare the incremental partial task execution capacity level of the first computing device to a threshold level;
in response to determining that the incremental partial task execution capacity level of the first computing device is above the threshold level, select one or more encoded data slices of a slice group stored by the first computing device in the local memory, wherein the one or more encoded data slices are additionally stored by a second computing device of the DSN;
obtain, from the second computing device via the network interface, at least one partial task associated with the one or more encoded data slices of the slice group; and
facilitate execution of the at least one partial task on the one or more encoded data slices of the slice group stored by the first computing device to produce partial results.
11 . The first computing device of claim 10 , wherein the processing module is further configured to:
send, via the network interface, the partial results to at least one of the second computing device or another device of the DSN associated with assignment of the at least one partial task.
12 . The first computing device of claim 10 , wherein the processing module is configured to obtain the at least one partial task associated with the one or more encoded data slices of the slice group by sending, via the network interface, a partial task retrieval request to the second computing device.
13 . The first computing device of claim 10 , wherein the processing module is configured to obtain the at least one partial task associated with the one or more encoded data slices of the slice group by:
retrieving, via the network interface, DST allocation information relating to the at least one partial task and data partition information regarding the one or more encoded data slices; and
updating, via the network interface, the DST allocation information to indicate that the second computing device is no longer affiliated with execution of the at least one partial task.
14 . The first computing device of claim 10 , wherein facilitating execution of the at least one partial task includes:
retrieving the one or more encoded data slices from the local memory; and
executing the at least one partial task on the retrieved one or more encoded data slices.
15 . The first computing device of claim 10 further comprises:
a local task queue, wherein facilitating execution of the at least one partial task includes updating the local task queue to include the at least one partial task.
16 . The first computing device of claim 10 , wherein the processing module is configured to select the one or more encoded data slices of the slice group by:
obtaining, via the network interface, a partial task to slice identifier (ID) list; and
based on the partial task to slice ID list, identifying the one or more encoded data slices of the slice group.
17 . The first computing device of claim 10 , wherein selecting the one or more encoded data slices of the slice group stored by the first computing device includes:
generating a partial task execution request requesting to execute at least one partial task relating to one or more encoded data slices that are stored in the first computing device; and
sending, via the network interface, the partial task execution request to the second computing device.
18 . The first computing device of claim 10 , wherein the at least one partial task includes an unexecuted partial task assigned to the second computing device.
19 . A computer readable storage medium comprises:
at least one memory section that stores operational instructions that, when executed by one or more processing modules of a first computing device of a dispersed storage network (DSN) having storage resources including a plurality computing devices, causes the first computing device to:
determine an incremental partial task execution capacity level of the first computing device;
compare the incremental partial task execution capacity level of the first computing device to a threshold level;
in response to determining that the incremental partial task execution capacity level of the first computing device is above the threshold level, select one or more encoded data slices of a slice group stored by the first computing device, wherein the one or more encoded data slices are additionally stored by a second computing device of the DSN;
obtain, from the second computing device, at least one partial task associated with the one or more encoded data slices of the slice group;
facilitate execution of the at least one partial task on the one or more encoded data slices of the slice group stored by the first computing device to produce partial results; and
send the partial results to the second computing device.
20 . The computer readable storage medium of claim 19 , wherein facilitating execution of the at least one partial task includes:
retrieving the one or more encoded data slices from a local memory of the first computing device; and
executing the at least one partial task on the retrieved one or more encoded data slices.