Storage unit partial task processing
A method for execution by one or more storage units of a dispersed storage network (DSN). The method begins by receiving, at a first storage unit, a request for a partial task. The method continues by generating a slice request, to one or more additional storage units, when the first storage unit does not contain all encoded data slices required to execute the partial task. The method continues by receiving the at least one additional encoded data slice from the one or more additional storage units and performing the partial task on the first encoded data slice and the at least one additional encoded data slice to produce at least partial results.
1. A method for execution by a storage unit, the method comprises:
receiving, at a first storage unit, a request for a partial task associated with a first encoded data slice;
generating a slice request, to one or more additional storage units, when the first storage unit does not contain all encoded data slices required to execute the partial task;
receiving at least one additional encoded data slice from the one or more additional storage units; and
performing the partial task on the first encoded data slice from the first storage unit and the at least one additional encoded data slice to produce at least partial results.
2. The method of claim 1 further comprises identifying a data record associated with the first encoded data slice based on one or more of: obtaining a slice name of the first encoded data slice, performing a data record identifier lookup in a slice name to data list, or extracting a data record identifier from the first encoded data slice.
3. The method of claim 2 , wherein it is determined that the data record includes the at least one additional encoded data slice by at least one of: performing a data record ID to slice name lookup, receiving a list of slice names, or a query.
4. The method of claim 1 , wherein the generating the slice request includes one or more of: identifying a slice name associated with the at least one additional encoded data slice, identifying the one or more additional storage units associated with the at least one additional encoded data slice, generating a partial task field entry that includes at least a portion of the partial task, or generating a credential field entry that includes a signature.
5. The method of claim 1 , wherein the performing the partial task on the first encoded data slice and the at least one additional encoded data slice to produce the at least partial results includes one or more of: aggregating at least a portion of the first encoded data slice and at least a portion of the at least one additional encoded data slice to produce a data record and executing at least a portion of the partial task on the data record to produce the partial results.
6. The method of claim 1 , wherein the first storage unit includes a distributed storage and task (DST) execution unit.
7. The method of claim 1 , wherein the one or more additional storage units each include a distributed storage and task (DST) execution unit.
8. The method of claim 1 , wherein the first storage unit and the one or more additional storage units each include a processing module.
9. A storage unit comprises:
an interface;
a local memory; and
a processing module operably coupled to the interface and the local memory, wherein the processing module functions to:
receive, at a first storage unit, a request for a partial task associated with a first encoded data slice;
generate a slice request, to one or more additional storage units, when the first storage unit does not contain all encoded data slices required to execute the partial task;
receive at least one additional encoded data slice from the one or more additional storage units; and
perform the partial task on the first encoded data slice from the first storage unit and the at least one additional encoded data slice to produce at least partial results.
10. The storage unit of claim 9 further comprises identifying a data record associated with the first encoded data slice based on one or more of: obtaining a slice name of the first encoded data slice, performing a data record identifier lookup in a slice name to data list, or extracting a data record identifier from the first encoded data slice.
11. The storage unit of claim 10 , wherein it is determined that the data record includes the at least one additional encoded data slice by at least one of: performing a data record ID to slice name lookup, receiving a list of slice names, or a query.
12. The storage unit of claim 9 , wherein the generate a slice request includes one or more of: identifying a slice name associated with the at least one additional encoded data slice, identifying the one or more additional storage units associated with the at least one additional encoded data slice, generating a partial task field entry that includes at least a portion of the partial task, or generating a credential field entry that includes a signature.
13. The storage unit of claim 9 , wherein the perform the partial task on the first encoded data slice and the at least one additional encoded data slice to produce the at least partial results includes one or more of: aggregating at least a portion of the first encoded data slice and at least a portion of the at least one additional encoded data slice to produce a data record and executing at least a portion of the partial task on the data record to produce the partial results.
14. The storage unit of claim 9 , wherein the first storage unit and the one or more additional storage units each include a processing module.
15. A system, the system comprises:
an interface;
a local memory; and
a processing module operably coupled to the interface and the local memory, wherein the processing module functions to:
receive, at a first storage unit, a request for a partial task associated with a first encoded data slice;
generate a slice request, to one or more additional storage units, when the first storage unit does not contain all encoded data slices required to execute the partial task;
receive at least one additional encoded data slice from the one or more additional storage units; and
perform the partial task on the first encoded data slice from the first storage unit and the at least one additional encoded data slice to produce at least partial results.
16. The system of claim 15 further comprises identify a data record associated with the first encoded data slice based on one or more of: obtaining a slice name of the first encoded data slice, performing a data record identifier lookup in a slice name to data list, or extracting a data record identifier from the first encoded data slice.
17. The system of claim 16 , wherein it is determined that the data record includes the at least one additional encoded data slice by at least one of: performing a data record ID to slice name lookup, receiving a list of slice names, or a query.
18. The system of claim 15 , wherein the generate a slice request includes one or more of: identifying a slice name associated with the at least one additional encoded data slice, identifying the one or more additional storage units associated with the at least one additional encoded data slice, generating a partial task field entry that includes at least a portion of the partial task, or generating a credential field entry that includes a signature.
19. The system of claim 15 , wherein the perform the partial task on the first encoded data slice and the at least one additional encoded data slice to produce the at least partial results includes one or more of: aggregating at least a portion of the first encoded data slice and at least a portion of the at least one additional encoded data slice to produce a data record and executing at least a portion of the partial task on the data record to produce the partial results.
20. The system of claim 15 , wherein the first storage unit and the one or more additional storage units each include a processing module.