READS FOR DISPERSED COMPUTATION JOBS
A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN). The method begins by receiving, by a first processing module, an encoded data slice and an associated partial task. The method continues by identifying a data record associated with the encoded data slice. The method continues by generating a slice request for a data record that includes at least one additional encoded data slice not received by the first processing module and outputting the slice request to a second processing module. The method continues by receiving, by the first processing module, the at least one additional encoded data slice from the second processing module and performing the associated partial task on the encoded data slice and the at least one additional encoded data slice to produce at least partial results.
1 . A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:
receiving, by a first processing module, an encoded data slice and an associated partial task;
identifying a data record associated with the encoded data slice;
generating a slice request for a data record that includes at least one additional encoded data slice not received by the first processing module;
outputting the slice request to a second processing module;
receiving, by the first processing module, the at least one additional encoded data slice from the second processing module; and
performing the associated partial task on the encoded data slice and the at least one additional encoded data slice to produce at least partial results.
2 . The method of claim 1 , wherein the identifying a data record associated with the encoded data slice is based on one or more of: obtaining a slice name of the encoded data slice, performing a data record identifier lookup in a slice name to data list, or extracting a data record identifier from the encoded data slice.
3 . The method of claim 1 , wherein it is determined that the data record includes the at least one additional encoded data slice by at least one of: performing a data record ID to slice name lookup, receiving a list of slice names, or a query.
4 . The method of claim 1 , wherein the generating the slice request includes one or more of: identifying a slice name associated with the at least one additional encoded data slice, identifying the second processing module associated with the at least one additional encoded data slice, generating a partial task field entry that includes at least a portion of the associated partial task, or generating a credential field entry that includes a signature.
5 . The method of claim 1 , wherein the performing the associated partial task on the encoded data slice and the at least one additional encoded data slice to produce at least partial results includes one or more of: aggregating at least a portion of the encoded data slice and at least a portion of the at least one additional encoded data slice to produce the data record and executing at least a portion of the associated partial task on the data record to produce the partial results.
6 . The method of claim 1 , wherein the second processing module includes a distributed storage and task (DST) execution unit.
7 . The method of claim 1 , wherein the second processing module is included as part of a storage unit.
8 . A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:
an interface;
a local memory; and
a processing module operably coupled to the interface and the local memory, wherein the processing module functions to:
receive an encoded data slice and an associated partial task;
identify a data record associated with the encoded data slice;
generate a slice request for a data record that includes at least one additional encoded data slice not received;
output the slice request to another processing module;
receive the at least one additional encoded data slice from the another processing module; and
perform the associated partial task on the encoded data slice and the at least one additional encoded data slice to produce at least partial results.
9 . The computing device of claim 8 , wherein the identify a data record associated with the encoded data slice is based on one or more of: obtaining a slice name of the encoded data slice, performing a data record identifier lookup in a slice name to data list, or extracting a data record identifier from the encoded data slice.
10 . The computing device of claim 8 , wherein it is determined that the data record includes the at least one additional encoded data slice by at least one of: performing a data record ID to slice name lookup, receiving a list of slice names, or a query.
11 . The computing device of claim 8 , wherein the generate a slice request includes one or more of: identifying a slice name associated with the at least one additional encoded data slice, identifying the another processing module associated with the at least one additional encoded data slice, generating a partial task field entry that includes at least a portion of the associated partial task, or generating a credential field entry that includes a signature.
12 . The computing device of claim 8 , wherein the perform the associated partial task on the encoded data slice and the at least one additional encoded data slice to produce at least partial results includes one or more of: aggregating at least a portion of the encoded data slice and at least a portion of the at least one additional encoded data slice to produce the data record and executing at least a portion of the associated partial task on the data record to produce the partial results.
13 . The computing device of claim 8 , wherein the another processing module includes a distributed storage and task (DST) execution unit.
14 . The computing device of claim 8 , wherein the another processing module is included as part of a storage unit.
15 . A system, the system comprises:
an interface;
a local memory; and
a processing module operably coupled to the interface and the local memory, wherein the processing module functions to:
receive an encoded data slice and an associated partial task;
identify a data record associated with the encoded data slice;
generate a slice request for a data record that includes at least one additional encoded data slice not received;
output the slice request to another processing module;
receive the at least one additional encoded data slice from the another processing module; and
perform the associated partial task on the encoded data slice and the at least one additional encoded data slice to produce at least partial results.
16 . The system of claim 15 , wherein the identify a data record associated with the encoded data slice is based on one or more of: obtaining a slice name of the encoded data slice, performing a data record identifier lookup in a slice name to data list, or extracting a data record identifier from the encoded data slice.
17 . The system of claim 15 , wherein it is determined that the data record includes the at least one additional encoded data slice by at least one of: performing a data record ID to slice name lookup, receiving a list of slice names, or a query.
18 . The system of claim 15 , wherein the generate a slice request includes one or more of: identifying a slice name associated with the at least one additional encoded data slice, identifying the another processing module associated with the at least one additional encoded data slice, generating a partial task field entry that includes at least a portion of the associated partial task, or generating a credential field entry that includes a signature.
19 . The system of claim 15 , wherein the perform the associated partial task on the encoded data slice and the at least one additional encoded data slice to produce at least partial results includes one or more of: aggregating at least a portion of the encoded data slice and at least a portion of the at least one additional encoded data slice to produce the data record and executing at least a portion of the associated partial task on the data record to produce the partial results.
20 . The system of claim 15 , wherein the another processing module is included as part of a storage unit.