Mapreduce implementation using an on-demand network code execution system

1. A system to utilize tasks in an on-demand code execution system as an implementation of the MapReduce programming model, the system comprising:

a non-transitory data store configured to store:

a map task, the map task corresponding to code executable by the on-demand code execution system to process a portion of a set of data to result in an output;

a reduce task, the reduce task corresponding to code executable by the on-demand code execution system to process a plurality of outputs from individual executions of the map task to result in an aggregated output; and

a coordinator task, the coordinator task corresponding to code executable by the on-demand code execution system to initiate executions of the map task and the reduce task;

one or more processors in communication with the non-transitory data store and configured with computer-executable instructions to:

obtain a request to analyze the set of data according to the implementation of the MapReduce programming model, wherein the request designates the map task and the reduce task;

initiate a first execution of the coordinator task within the on-demand code execution system, wherein the first execution of the coordinator task causes the on-demand code execution system to:

assign individual portions of the set of data to individual executions of the map task;

implement the individual executions of the map task to process the individual portions of the set of data and result in the plurality of outputs;

establish an event trigger for a second execution of the coordinator task, the event trigger requesting that the second execution of the coordinator task be initiated in response to completion of the individual executions of the map task; and

halt the first execution of the coordinator task;

detect completion of the individual executions of the map task; and

in response to completion of the individual executions of the map task, initiate the second execution of the coordinator task within the on-demand code execution system, wherein the second execution of the coordinator task causes the on-demand code execution system to:

implement at least one reduce task to process the plurality of outputs from the individual executions of the map task to result in an aggregated output; and

return the aggregated output as a result of analysis of the set of data.

2. The system of claim 1 , wherein the coordinator task comprises a first coordinator task corresponding to code executable by the on-demand code execution system to initiate executions of the map task and a second coordinator task corresponding to code executable by the on-demand code execution system to initiate executions of the reduce task, wherein the first execution of the coordinator task corresponds to an execution of the first coordinator task, and wherein the second execution of the coordinator task corresponds to an execution of the second coordinator task.

3. The system of claim 1 , wherein the event trigger requests that an additional execution of the coordinator task be initiated in response to completion of any one of the individual executions of the map task, and wherein the one or more processors in are further configured with computer-executable instructions to:

detect completion of a first execution of the map task; and

initiate the additional execution of the coordinator task within the on-demand code execution system, wherein the additional execution of the coordinator task causes the on-demand code execution system to detect that a second execution of the map task has not yet completed and to halt the additional execution of the coordinator task.

4. The system of claim 1 , wherein the at least one execution of the reduce task comprises multiple executions of the reduce task, the multiple executions comprising:

a plurality of first executions of the reduce task to process the plurality of outputs from the individual executions of the map task to result in a plurality of intermediate aggregated outputs; and

at least one second execution of the reduce task to process the plurality of intermediate aggregated outputs from the plurality of first executions of the reduce task to result the aggregated output.

5. A computer-implemented method to analyze a set of data utilizing tasks in an on-demand code execution system, the computer-implemented method comprising:

obtaining a request to analyze the set of data, wherein the request designates:

a map task corresponding to code executable by the on-demand code execution system to process a portion of the set of data to result in an output; and

a reduce task corresponding to code executable by the on-demand code execution system to process a plurality of outputs from individual executions of the map task to result in an aggregated output; and

initiating a first execution of a coordinator task within the on-demand code execution system, wherein the first execution of the coordinator task causes the on-demand code execution system to:

assign individual portions of the set of data to individual executions of the map task;

implement the individual executions of the map task to process the individual portions of the set of data and result in the plurality of outputs;

establish an event trigger for a second execution of the coordinator task, the event trigger requesting that the second execution of the coordinator task be initiated in response to completion of the individual executions of the map task; and

halt the first execution of the coordinator task;

detecting completion of the individual executions of the map task; and

in response to completion of the individual executions of the map task, initiating the second execution of the coordinator task within the on-demand code execution system, wherein the second execution of the coordinator task causes the on-demand code execution system to:

implement at least one execution of the reduce task to process the plurality of outputs from the individual executions of the map task to result in an aggregated output; and

return the aggregated output as a result of analysis of the set of data.

6. The computer-implemented method of claim 5 , wherein the request includes the map task and the request task.

7. The computer-implemented method of claim 5 , wherein execution of the coordinator task causes the on-demand code execution system to assign individual portions of the set of data to individual executions of the map task based at least in part on a memory expected to be utilized during an individual execution of the map task to process an individual portion of the set of data.

8. The computer-implemented method of claim 7 , wherein the memory expected to be utilized during an individual execution of the map task to process an individual portion of the set of data is determined based at least in part on a parameter provided with the request to analyze the set of data or a parameter maintained by the on-demand code execution system.

9. The computer-implemented method of claim 8 , wherein the parameter maintained by the on-demand code execution system is determined based at least in part on a prior execution of the map task.

10. The computer-implemented method of claim 5 , wherein the coordinator task comprises a first coordinator task corresponding to code executable by the on-demand code execution system to initiate executions of the map task and a second coordinator task corresponding to code executable by the on-demand code execution system to initiate executions of the reduce task, wherein the first execution of the coordinator task corresponds to an execution of the first coordinator task, and wherein the second execution of the coordinator task corresponds to an execution of the second coordinator task.

11. Non-transitory computer readable media comprising instructions executable by an on-demand code execution system to analyze a set of data, wherein execution of the instructions cause the on-demand code execution system to:

obtain a request to analyze the set of data, wherein the request designates:

a map task corresponding to code executable by the on-demand code execution system to process a portion of the set of data to result in an output; and

a reduce task corresponding to code executable by the on-demand code execution system to process a plurality of outputs from individual executions of the map task to result in an aggregated output; and

initiate a first execution of a coordinator task within the on-demand code execution system, wherein the first execution of the coordinator task causes the on-demand code execution system to:

assign individual portions of the set of data to individual executions of the map task;

implement the individual executions of the map task to process the individual portions of the set of data and result in the plurality of outputs;

establish an event trigger for a second execution of the coordinator task, the event trigger requesting that the second execution of the coordinator task be initiated in response to completion of the individual executions of the map task; and

halt the first execution of the coordinator task;

detect completion of the individual executions of the map task; and

in response to completion of the individual executions of the map task, initiate the second execution of the coordinator task within the on-demand code execution system, wherein the second execution of the coordinator task causes the on-demand code execution system to:

implement at least one execution of the reduce task to process the plurality of outputs from the individual executions of the map task to result in an aggregated output; and

return the aggregated output as a result of analysis of the set of data.

12. The non-transitory computer-readable media of claim 11 , wherein the request references the map task and the request task as tasks pre-existing on the on-demand code execution system.

13. The non-transitory computer-readable media of claim 11 , wherein the first execution of the coordinator task causes the on-demand code execution system to implement the individual executions of the map task at least in part in serial, and wherein data regarding a first execution of the map task is utilized by the first execution of the coordinator task to modify an assignment of an individual portion of the set of data with respect to a second execution of the map task.

14. The non-transitory computer-readable media of claim 11 , wherein the coordinator task comprises a first coordinator task corresponding to code executable by the on-demand code execution system to initiate executions of the map task and a second coordinator task corresponding to code executable by the on-demand code execution system to initiate executions of the reduce task, wherein the first execution of the coordinator task corresponds to an execution of the first coordinator task, and wherein the second execution of the coordinator task corresponds to an execution of the second coordinator task.

15. The non-transitory computer-readable media of claim 11 , wherein execution of the coordinator task causes the on-demand code execution system to assign individual portions of the set of data to individual executions of the map task based at least in part on a memory expected to be utilized during an individual execution of the map task to process an individual portion of the set of data.

16. The non-transitory computer-readable media of claim 15 , wherein the memory expected to be utilized during an individual execution of the map task to process an individual portion of the set of data is determined based at least in part on a parameter provided with the request to analyze the set of data or a parameter maintained by the on-demand code execution system.

17. The non-transitory computer-readable media of claim 16 , wherein the parameter maintained by the on-demand code execution system is determined based at least in part on a prior execution of the map task.