Serial storage node processing of data functions

1. A system, comprising:

a first plurality of storage nodes, wherein:

each storage node of the first plurality of storage nodes includes a non-transitory storage medium configured to store at least one erasure encoded symbol for a data unit;

the first plurality of storage nodes is configured to receive a data function request targeting the data unit with a serial-function;

at least one storage node of the first plurality of storage nodes is configured to:

determine at least a portion of the serial-function targeting the data unit;

receive a prior intermediate context from a prior storage node of the first plurality of storage nodes;

decode a second erasure encoded symbol from the non-transitory storage medium into a decoded second symbol for the data unit;

identify a decoded second target subunit from the decoded second symbol;

process, using the serial-function, the decoded second target subunit and the prior intermediate context to generate a next intermediate context; and

send the next intermediate context to a next storage node of the first plurality of storage nodes;

a first storage node of the first plurality of storage nodes, wherein the first storage node includes a first non-transitory storage medium and is configured to:

decode a first erasure encoded symbol from the first non-transitory storage medium into a decoded first symbol for a data unit, wherein the decoded first symbol includes a decoded first target subunit of the data unit;

identify the decoded first target subunit in the decoded first symbol;

process the decoded first target subunit using the serial-function to generate a first intermediate context; and

send the first intermediate context to the next storage node of the first plurality of storage nodes as a first prior intermediate context; and

a final storage node of the first plurality of storage nodes, wherein the final storage node includes a second non-transitory storage medium and is configured to:

decode a final erasure encoded symbol from the second non-transitory storage medium into a decoded final symbol for the data unit, wherein the decoded final symbol includes a decoded final target subunit of the data unit;

identify the decoded final target subunit in the decoded final symbol;

process the decoded final target subunit and a final prior intermediate context using the serial-function to generate a serial-function result for the data unit; and

return the serial-function result.

2. The system of claim 1 , wherein the serial-function includes:

a map-function, wherein the map-function is configured to generate a temporary intermediate context for the decoded final target subunit; and

a reduce-function, wherein the reduce-function is configured to generate the serial-function result from the temporary intermediate context and the final prior intermediate context.

3. The system of claim 1 , wherein at least one storage node of the first plurality of storage nodes is further configured to:

receive at least a portion of the serial-function from the prior storage node of the first plurality of storage nodes; and

send at least a portion of the serial-function to the next storage node of the first plurality of storage nodes.

4. The system of claim 1 , further comprising:

a second plurality of storage nodes configured to:

store a plurality of erasure encoded symbols corresponding to a prior data unit targeted by the serial-function;

generate, using the serial-function, at least one intermediate context from an erasure encoded symbol of the plurality of erasure encoded symbols corresponding to the prior data unit; and

send a prior data unit intermediate context to the first storage node of the first plurality of storage nodes, wherein the first storage node is further configured to use the prior data unit intermediate context to process the decoded first target subunit.

5. The system of claim 1 , further comprising:

an incomplete subunit processor configured to:

receive an incomplete subset of intermediate contexts from the first plurality of storage nodes, wherein the incomplete subset of intermediate contexts includes incomplete subunits;

aggregate at least one complete target subunit from the incomplete subunits;

process the at least one complete target subunit using the serial-function and a prior intermediate context to generate at least one additional intermediate context; and

send the at least one additional intermediate context to the next storage node including a next symbol for the data unit.

6. The system of claim 1 , wherein:

the decoded first symbol further includes a decoded first incomplete subunit;

the first intermediate context further includes the decoded first incomplete subunit; and

the next storage node of the first plurality of storage nodes is configured to identify a decoded next target subunit in a decoded next symbol from a combination of the decoded next symbol and the decoded first incomplete subunit.

7. The system of claim 1 , wherein:

at least one storage node of the first plurality of storage nodes includes a peer communication channel to the prior storage node and the next storage node of the first plurality of storage nodes; and

at least one storage node of the first plurality of storage nodes is further configured to:

receive, from the prior storage node, the prior intermediate context via the peer communication channel; and

send, to the next storage node, the next intermediate context via the peer communication channel.

8. The system of claim 1 , wherein:

the at least one erasure encoded symbol of each storage node of the first plurality of storage nodes is an ordered symbol among a plurality of symbols corresponding to the data unit, wherein the plurality of symbols have a symbol order;

the decoded first symbol is a first symbol in the symbol order;

the final decoded symbol is a terminal symbol in the symbol order; and

the serial-function result from the final storage node includes a final result returned to a client system.

9. The system of claim 1 , further comprising:

a client request handler configured to:

receive the data function request;

identify a function data set including the data unit, wherein the first storage node and the final storage node are among the first plurality of storage nodes; and

return a function result based on the serial-function result to a client system, wherein the client system is not among the first plurality of storage nodes.

10. A computer-implemented method, comprising:

determining, based on a data function request, a serial-function;

decoding, at a first storage node including a first non-transitory storage medium, a first erasure encoded symbol from the first non-transitory storage medium into a decoded first symbol for a data unit, wherein the decoded first symbol includes a decoded first target subunit of the data unit;

identifying, at the first storage node, the decoded first target subunit in the decoded first symbol; and

processing, at the first storage node, the decoded first target subunit using the serial-function to generate a first intermediate context, wherein the first intermediate context includes a first function result;

sending the first intermediate context to a next storage node of a first plurality of storage nodes as a first prior intermediate context;

receiving, at each next storage node of the first plurality of storage nodes, a prior intermediate context from a prior storage node of the first plurality of storage nodes;

decoding, at that next storage node of the first plurality of storage nodes and from a non-transitory storage medium of that next storage node, a second erasure encoded symbol into a decoded second symbol for the data unit;

identifying, at that next storage node, a decoded second target subunit from the decoded second symbol;

processing, using the serial-function, the decoded second target subunit and the prior intermediate context to generate a next intermediate context;

sending the next intermediate context to another next storage node of the first plurality of storage nodes to provide the prior intermediate context for that another next storage node;

decoding, at a final storage node including a second non-transitory storage medium, a final erasure encoded symbol from the second non-transitory storage medium into a decoded final symbol for the data unit, wherein the decoded final symbol includes a decoded final target subunit of the data unit;

identifying, at the final storage node, the decoded final target subunit in the decoded final symbol;

processing, at the final storage node, the decoded final target subunit and a final prior intermediate context using the serial-function to generate a serial-function result; and

returning the serial-function result.

11. The computer-implemented method of claim 10 , further comprising:

receiving, by at least one storage node of the first plurality of storage nodes, at least a portion of the serial-function from the prior storage node of the first plurality of storage nodes; and

sending, by at least one storage node of the first plurality of storage nodes, at least a portion of the serial-function to the next storage node of the first plurality of storage nodes.

12. The computer-implemented method of claim 10 , further comprising:

storing, by a second plurality of storage nodes, a plurality of erasure encoded symbols corresponding to a prior data unit targeted by the serial-function;

generating, using the serial-function, at least one intermediate context from an erasure encoded symbol of the plurality of erasure encoded symbols corresponding to the prior data unit; and

sending a prior data unit intermediate context to the first storage node of the first plurality of storage nodes, wherein the first storage node is further configured to use the prior data unit intermediate context to process the decoded first target subunit.

13. The computer-implemented method of claim 10 , further comprising:

identifying an incompletely recovered erasure encoded symbol from a failed storage node decode operation in the first plurality of storage nodes;

executing a recovery process for the incompletely recovered erasure encoded symbol to determine a recovered subunit of the data unit;

processing the recovered subunit using the serial-function and a prior intermediate context to generate at least one additional intermediate context; and

sending the at least one additional intermediate context to the next storage node including a next symbol for the data unit.

14. The computer-implemented method of claim 10 , further comprising:

receiving an incomplete subset of intermediate contexts from the first plurality of storage nodes, wherein the incomplete subset of intermediate contexts includes incomplete subunits;

aggregating at least one complete target subunit from the incomplete subunits;

processing the at least one complete target subunits using the serial-function and a prior intermediate context to generate at least one additional intermediate context; and

sending the at least one additional intermediate context to the next storage node including a next symbol for the data unit.

15. The computer-implemented method of claim 10 , wherein:

the decoded first symbol further includes a decoded first incomplete subunit;

the first intermediate context further includes the decoded first incomplete subunit; and

the next storage node of the first plurality of storage nodes identifies a decoded next target subunit in a decoded next symbol from a combination of the decoded next symbol and the decoded first incomplete subunit.

16. The computer-implemented method of claim 10 , wherein:

at least one storage node of the first plurality of storage nodes includes a peer communication channel to the prior storage node and the next storage node of the first plurality of storage nodes;

receiving, by at least one storage node of the first plurality of storage nodes, the prior intermediate context is via the peer communication channel; and

sending, by at least one storage node of the first plurality of storage nodes, the next intermediate context is via the peer communication channel.

17. The computer-implemented method of claim 10 , wherein:

at least one erasure encoded symbol of each storage node of the first plurality of storage nodes is an ordered symbol among a plurality of symbols corresponding to the data unit, wherein the plurality of symbols have a symbol order;

the decoded first symbol is a first symbol in the symbol order;

the final decoded symbol is a terminal symbol in the symbol order; and

the serial-function result from the final storage node includes a final result returned to a client system.

18. The computer-implemented method of claim 10 , further comprising:

receiving the data function request;

identifying a function data set including the data unit, wherein the first storage node and the final storage node are among the first plurality of storage nodes; and

returning a function result based on the serial-function result to a client system, wherein the client system is not among the first plurality of storage nodes.

19. A system, comprising:

a plurality of storage nodes, wherein each storage node of the plurality of storage nodes includes a non-transitory storage medium configured to store at least one erasure encoded symbol from a plurality of erasure encoded symbols for a data unit, and including:

a first storage node including a first non-transitory storage medium; and

a final storage node including a second non-transitory storage medium;

means for determining, based on a data function request, a serial-function;

means for decoding, at the first storage node, a first erasure encoded symbol from the first non-transitory storage medium into a decoded first symbol for a data unit, wherein the decoded first symbol includes a decoded first target subunit of the data unit;

means for identifying, at the first storage node, the decoded first target subunit in the decoded first symbol;

means for processing, at the first storage node, the decoded first target subunit using the serial-function to generate a first intermediate context;

means for sending the first intermediate context to a next storage node of the plurality of storage nodes as a first prior intermediate context;

means for receiving, at each next storage node of the plurality of storage nodes, a prior intermediate context from a prior storage node of the plurality of storage nodes;

means for decoding, at that next storage node of the plurality of storage nodes and from a non-transitory storage medium of that next storage node, a second erasure encoded symbol into a decoded second symbol for the data unit;

means for identifying, at that next storage node, a decoded second target subunit from the decoded second symbol;

means for processing, using the serial-function, the decoded second target subunit and the prior intermediate context to generate a next intermediate context;

means for sending the next intermediate context to another next storage node of the plurality of storage nodes to provide the prior intermediate context for that another next storage node;

means for decoding, at the final storage node, a final erasure encoded symbol from the second non-transitory storage medium into a decoded final symbol for the data unit, wherein the decoded final symbol includes a decoded final target subunit of the data unit;

means for identifying, at the final storage node, the decoded final target subunit in the decoded final symbol;

means for processing, at the final storage node, the decoded final target subunit and a final prior intermediate context using the serial-function to generate a serial-function result; and

means for returning the serial-function result.

20. The system of claim 19 , wherein:

the decoded first symbol further includes a decoded first incomplete subunit;

the first intermediate context further includes the decoded first incomplete subunit; and

the means for identifying, at the next storage node of the plurality of storage nodes, identifies a decoded next target subunit in the decoded next symbol from a combination of the decoded next symbol and the decoded first incomplete subunit.