MULTI-TASK PROCESSING IN A DISTRIBUTED STORAGE NETWORK

1 . A method comprises:

temporarily storing, by a computing device of a dispersed storage network (DSN), a plurality of tasks in a task queue to produce queued tasks;

identifying, by the computing device, a task of the queued tasks for execution, wherein the task corresponds to performing a particular function on data, wherein the data is partitioned into a set of partial data elements, wherein a first partial data element of the set of partial data elements is storage in a first storage unit of a set of storage units of the DSN, wherein the first storage unit includes a task execution module;

partitioning, by the computing device, the task into a plurality of partial tasks;

sending, by the computing device, partial task execution requests to at least some of the set of storage units, wherein a first one of the partial task execution requests is sent to the first storage unit and includes a first partial task of the set of the plurality of partial tasks and a data access request regarding the first partial data element;

transferring, by the computing device, the task from the task queue to a task in process index and establishing an expiration time; and

when a partial task of the plurality of partial tasks has not been completed prior to the expiration time, transferring, by the computing device, the task from the task in process index to the task queue indicating that the task was not completed prior to the expiration time and re-queuing execution of at least a portion of the task.

2 . The method of claim 1 , wherein the identifying the task comprises one of:

utilizing a first in first out approach;

utilizing a task requester based priority scheme;

utilizing a task priority based scheme;

utilizing a resource balancing selection scheme; and

utilizing a conflict avoidance scheme.

3 . The method of claim 1 further comprises:

the data is partitioned is partitioned to the set of partial data elements in accordance with a dispersed storage error encoding using an encoded matrix that includes a unity matrix component, wherein a data segment of the data is encoded into a set of encoded data slices.

4 . The method of claim 1 further comprises:

when each of the plurality of partial tasks has been completed prior to the expiration time, deleting, by the computing device, the task from the task in process index indicating that the task has been successfully completed.

5 . The method of claim 1 further comprises:

prior to conclusion of the expiration time, receiving a request from one of the least some of the set of storage units, wherein the request is an indication the one of the least some of the set of storage units is processing a corresponding one of the plurality of partial tasks but requires addition time beyond the expiration time to complete;

changing the expiration time to a new expiration time; and

sending the new expiration time to the one of the least some of the set of storage units.

6 . The method of claim 1 further comprises:

temporarily storing, by the first storage unit, a plurality of first partial tasks corresponding to a first partial task of each the plurality of tasks in a first storage unit task queue to produce queued first partial tasks;

receiving, by the first storage unit, a request to perform a first partial task of the task on the first partial data element;

transferring, by the first storage unit, the first partial task from the first storage unit task queue to a first storage unit task in process index;

determining, by the storage unit, whether the first storage unit has capacity to at least start performing of the first partial task of the task on the first partial data element prior to conclusion of the expiration time;

when the first storage unit has capacity to complete performance of the first partial task of the task on the first partial data element prior to conclusion of the expiration time, performing, by the first storage unit, the first partial task of the task on the first partial data element to produce a first partial result;

sending, by the first storage unit, the first partial result for the first partial task of the task to the computing device; and

deleting, by the first storage unit, the first partial task from the first storage unit task in process index.

7 . The method of claim 6 further comprises:

determining, by the first storage unit, that the first storage unit cannot complete performance of the first partial task of the task on the first partial data element prior to conclusion of the expiration time; and

prior to the conclusion of the expiration time, sending, by the first storage unit, a request to the computing device for an extension of the expiration time.

8 . The method of claim 6 further comprises:

determining, by the first storage unit, that the first storage unit cannot commence performance of the first partial task of the task on the first partial data element prior to conclusion of the expiration time; and

prior to the conclusion of the expiration time, sending, by the first storage unit, a notice to the computing device regarding the non-commencement of the performance of the first partial task of the task on the first partial data element.

9 . The method of claim 6 further comprises:

determining, by the first storage unit, that the first storage unit cannot commence performance of the first partial task of the task on the first partial data element prior to the conclusion of the expiration time; and

allowing, by the first storage unit, the conclusion of the expiration time without notice to the computing device.

10 . A computer readable memory comprises:

a first memory element that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN), causes the computing device to:

temporarily store a plurality of tasks in a task queue to produce queued tasks;

a second memory element that stores operational instructions that, when executed by the computing device, causes the computing device to:

identify a task of the queued tasks for execution, wherein the task corresponds to performing a particular function on data, wherein the data is partitioned into a set of partial data elements, wherein a first partial data element of the set of partial data elements is storage in a first storage unit of a set of storage units of the DSN, wherein the first storage unit includes a task execution module;

partition the task into a plurality of partial tasks;

send partial task execution requests to at least some of the set of storage units, wherein a first one of the partial task execution requests is sent to the first storage unit and includes a first partial task of the set of the plurality of partial tasks and a data access request regarding the first partial data element; and

transfer the task from the task queue to a task in process index and establishing an expiration time; and

a third memory element that stores operational instructions that, when executed by the computing device, causes the computing device to:

when a partial task of the plurality of partial tasks has not been completed prior to the expiration time, transfer the task from the task in process index to the task queue indicating that the task was not completed prior to the expiration time and re-queuing execution of at least a portion of the task.

11 . The computer readable memory of claim 10 , wherein the identifying the task comprises one of:

utilizing a first in first out approach;

utilizing a task requester based priority scheme;

utilizing a task priority based scheme;

utilizing a resource balancing selection scheme; and

utilizing a conflict avoidance scheme.

12 . The computer readable memory of claim 10 further comprises:

the data is partitioned is partitioned to the set of partial data elements in accordance with a dispersed storage error encoding using an encoded matrix that includes a unity matrix component, wherein a data segment of the data is encoded into a set of encoded data slices.

13 . The computer readable memory of claim 10 further comprises:

a fourth memory element that stores operational instructions that, when executed by the computing device, causes the computing device to:

when each of the plurality of partial tasks has been completed prior to the expiration time, delete the task from the task in process index indicating that the task has been successfully completed.

14 . The computer readable memory of claim 10 further comprises:

a fourth memory element that stores operational instructions that, when executed by the computing device, causes the computing device to:

prior to conclusion of the expiration time, receive a request from one of the least some of the set of storage units, wherein the request is an indication the one of the least some of the set of storage units is processing a corresponding one of the plurality of partial tasks but requires addition time beyond the expiration time to complete;

change the expiration time to a new expiration time; and

send the new expiration time to the one of the least some of the set of storage units.

15 . The computer readable memory of claim 10 further comprises:

a fourth memory element that stores operational instructions that, when executed by the first storage unit, causes the first storage unit to:

temporarily store a plurality of first partial tasks corresponding to a first partial task of each the plurality of tasks in a first storage unit task queue to produce queued first partial tasks;

receive a request to perform a first partial task of the task on the first partial data element;

transfer the first partial task from the first storage unit task queue to a first storage unit task in process index;

determine whether the first storage unit has capacity to at least start performing of the first partial task of the task on the first partial data element prior to conclusion of the expiration time;

when the first storage unit has capacity to complete performance of the first partial task of the task on the first partial data element prior to conclusion of the expiration time, perform the first partial task of the task on the first partial data element to produce a first partial result;

send the first partial result for the first partial task of the task to the computing device; and

delete the first partial task from the first storage unit task in process index.

16 . The computer readable memory of claim 15 further comprises:

a fifth memory element that stores operational instructions that, when executed by the first storage unit, causes the first storage unit to:

determine that the first storage unit cannot complete performance of the first partial task of the task on the first partial data element prior to conclusion of the expiration time; and

prior to the conclusion of the expiration time, send a request to the computing device for an extension of the expiration time.

17 . The computer readable memory of claim 15 further comprises:

a fifth memory element that stores operational instructions that, when executed by the first storage unit, causes the first storage unit to:

determine that the first storage unit cannot commence performance of the first partial task of the task on the first partial data element prior to conclusion of the expiration time; and

prior to the conclusion of the expiration time, send a notice to the computing device regarding the non-commencement of the performance of the first partial task of the task on the first partial data element.

18 . The computer readable memory of claim 15 further comprises:

a fifth memory element that stores operational instructions that, when executed by the first storage unit, causes the first storage unit to:

determine that the first storage unit cannot commence performance of the first partial task of the task on the first partial data element prior to the conclusion of the expiration time; and

allow the conclusion of the expiration time without notice to the computing device.