AUTOMATED EXPERIMENTATION PLATFORM

1 . An automated experimentation platform comprising:

one or more processors;

one or more memories;

one or more data-storage devices; and

computer instructions stored in one or more of the memories and data-storage devices that, when executed on one or more or the one or more processors, control the automated experimentation platform to

provide a visual integrated development environment through which workflows comprising input data sets, execution modules, intermediate data sets, and output data sets linked together in a graph are created and displayed; and

execute workflows to produce output data sets.

2 . The automated experimentation platform of claim 1 wherein the integrated development environment provides a dashboard display that includes:

a workflow-display panel;

input features through which data sets and execution modules are searched for among data sets and execution modules stored within the automated experimentation platform and incorporated into a workflow;

input features through which data sets and execution modules are downloaded from remote computer systems and incorporated into a workflow;

input features through which data sets, execution modules, and workflows are uploaded for storage within the automated experimentation platform; and

manipulation features through which data sets and execution modules are combined together to generate a workflow.

3 . The automated experimentation platform of claim 1 wherein the integrated development environment provides a dashboard display that includes:

a workflow-display panel that

displays representations of execution modules, representations of data sets, and links between data sets and execution modules; and

provides a workspace in which workflow graphs are assembled from representations of data sets, representations of execution modules, representations of input data sets, and representations of output data sets.

4 . The automated experimentation platform of claim 1 wherein a workflow represents an experiment, comprising one or more execution modules, that, when launched as a result of input to the visual integrated development environment, is executed by automated experimentation platform by executing the execution modules in one or more sequential execution phases during each of which one or more execution modules are scheduled for execution.

5 . The automated experimentation platform of claim 4 wherein, during an execution phase, one or more execution modules are scheduled for execution by the automated experimentation platform by distributing the one or more execution modules among available execution cluster nodes for parallel execution.

6 . The automated experimentation platform of claim 4 wherein the data sets input to the one or more execution modules that are scheduled for execution during an execution phase are either data sets initially input to the experiment or data sets that have been generated during execution of one or more preceding execution phases.

7 . The automated experimentation platform of claim 4 wherein a workflow is an acyclic graph in which one or more input data sets are linked to one or more execution modules, execution modules are associated with one or more intermediate or output data sets, and intermediate data sets are linked to execution modules.

8 . The automated experimentation platform of claim 1 further comprising:

a front end that comprises multiple front-end experiment dashboard applications that each executes on a user computer or other processor-controlled user device; and

a back end connected to the front end by a communications infrastructure including one or more of the Internet, personal area networks, local area networks, wide area networks, and communications sub-systems, the back end comprising one or more application-program-interface servers, a distributed catalog service, a cluster-management service, and multiple execution-cluster nodes implemented within one or more of one or more cloud-computing systems, one or more private data centers, and one or more aggregations of server computers, network-attached storage systems, internal networks, and main frames or other large computer systems.

9 . The automated experimentation platform of claim 8 wherein the front-end experiment dashboards exchange information, requests, and responses with the application-program-interface servers using a RESTful communications model.

10 . The automated experimentation platform of claim 8 wherein the application-program-interface servers

receive requests from, and send responses to, the front-end experiment-dashboard applications running on user computers;

carry out received requests by accessing services provided by the catalog service and cluster-management service; and

provide services to the execution cluster nodes and cluster-management service.

11 . The automated experimentation platform of claim 8

wherein the automated experimentation platform stores

execution modules, executable sets of computer instructions and configuration data,

experiments, complex computational tasks represented by workflows that combine data sets and executable modules,

data sets, and

jobs, execution instances of execution modules; and

wherein the catalog service provides an interface to the stored execution modules, the stored experiments, the stored data sets, and the stored jobs.

12 . The automated experimentation platform of claim 8 wherein the cluster-management service

receives, from the application-program-interface servers, job identifiers for jobs that need to be executed on the execution cluster nodes in order to carry out experiments; and

dispatches the jobs to appropriate execution cluster nodes for execution.

13 . The automated experimentation platform of claim 12 wherein the cluster-management service

forwards jobs that are ready for execution to particular execution cluster nodes for immediate execution; and

forwards jobs that need to wait for data produced by currently executing jobs or jobs waiting for execution to pinger routines executing within execution cluster nodes, the pinger routines intermittently checking for satisfaction of dependencies in order to launch jobs for which dependencies have been satisfied.

14 . The automated experimentation platform of claim 12 wherein, when a job has finished execution, the execution cluster node on which the job executed transmits output data and status information, via an application-program-interface server, to the catalog.

15 . The automated experimentation platform of claim 8 wherein workflows that represent experiments are textually encoded in the JavaScript object notation.

16 . The automated experimentation platform of claim 15 wherein a JavaScript-object-notation-encoded workflow comprises a list of JavaScript-object-notation-encoded execution modules, each including

an execution-module name;

a version number; and

encodings for each of one or more execution-module instances.

17 . The automated experimentation platform of claim 16 wherein a JavaScript-object-notation-encoded execution-module instance includes:

an instance name or identifier; and

a list or set of key-value pairs, the key-value pairs encoding indications of the data inputs to the execution module, data outputs from the execution module, static parameters, and variable parameters for the execution module.

18 . The automated experimentation platform of claim 1 wherein the integrated development environment provides cloning an experiment stored by the automated experimentation platform.

19 . The automated experimentation platform of claim 18 wherein an experiment is cloned by:

identifying, by searches of experiment stored by automated experimentation platform, an experiment;

displaying a representation of the experiment by the integrated development environment;

modifying the experiment by one or more of changing data sources, adding execution modules, deleting execution modules, changing execution modules, and altering data-flow links between execution modules, adding instances of execution modules, and deleting instances of execution modules.

20 . The automated experimentation platform of claim 19 wherein, because information about, and data sets produced by, previously executed experiments and instances of execution modules is maintained by the automated experimentation platform, instances of execution modules associated with the modified experiment that receive the same input as identical jobs in previously executed experiments need not be again executed during execution of the modified experiment.