SYSTEM & METHOD FOR COMPUTATIONALLY EFFICIENT AND STATISTICALLY ROBUST DESIGN OF MULTI-ARM MULTI-STAGE EXPERIMENTS
A method and system providing an improved technology for efficiently computing experimental design in strict statistically controlled settings. The method and system utilize a unique combination of statistical algorithms for designing, comparing the performance, and conducting multi-arm, multistage experiments efficiently, while radically reducing the computation time required on a personal computing device, thereby improving computer technology and performance. In particular, the method and system enable personal computing devices to compute very large multi-arm multistage experiments in a significantly reduced period of time. The present invention allows experimentalists to examine in a single statistically controlled experiment a vastly greater range of experimental options, with the possibility of dropping uninformative arms of the experiment, changing goals and parameters during the course of the experiment.
1 . A method of designing multi-arm multistage (MAMS) experiments, the method comprising:
identifying a MAMS experiment problem;
breaking the MAMS experiment problem into a plurality of independent increments, each independent increment comprising a multi-stage problem;
reducing a dimensionality of the plurality of independent increments using a Score Statistic;
transforming the MAMS experiment problem into a finite integration; and
computing the integration using a quasi-Monte Carlo approach.
2 . The method of claim 1 , wherein the method complexity increases linearly based on number of stages and number of arms, as represented by the equation J×D, where J is number of stages and D is number of arms.
3 . The method of claim 1 , wherein the method complexity does not increase exponentially based on number of stages and number of arms.
4 . The method of claim 1 , wherein breaking the MAMS experiment problem into a plurality of independent increments comprises collecting data from the MAMS experiment, and calculating a cumulative score test statistic.
5 . The method of claim 1 , wherein reducing a dimensionality of the plurality of independent increments using a Score Statistic comprises converting the MAMS experiment problem into J consecutive stages.
6 . The method of claim 1 , wherein transforming the MAMS experiment problem into a finite integration comprises transforming a correlated normal integration into an integration of independent normal variables, transforming a lower integration limits from negative infinity to zero.
7 . The method of claim 1 , wherein computing the integration using a quasi-Monte Carlo approach comprises generating lattice points of a lattice, shifting the lattice by a random vector amount, converting to points between zero and one, and evaluating integrand and taking an average.
8 . The method of claim 1 , wherein the computations necessary to create a multi-arm multistage (MAMS) experiments are made iteratively to create many designs for a range of design assumptions, such ranges and parameters being input by a user of the system either from an electronic file or manually.
9 . A system for multistage multi-arm (MAMS) experiments, the system comprising:
a parameter intake, configured to prompt and receive input for experiment design parameters;
a boundaries intake, configured to prompt and receive input for experiment design boundaries;
a simulator engine, configured to receive input parameters comprised of the experiment design parameters and the experiment design boundaries, and further configured to execute simulations determining boundaries efficacy based on the input parameters;
wherein the simulator engine utilizes a method comprising:
breaking a MAMS experiment problem into a plurality of independent increments, each independent increment comprising a three-stage problem;
reducing a dimensionality of the plurality of independent increments using a Score Statistic;
transforming the MAMS experiment problem into a finite integration; and
using a quasi-Monte Carlo number theoretic approach, computing the integration.
10 . The system of claim 9 , wherein the experiment design parameters comprise acceptable error rates, anticipated difference between arms, estimated variance of outcome, and allocation of sample between arms.
11 . The system of claim 9 , wherein the experiment design boundaries comprise select spending function and input spacing of interim looks.
12 . The system of claim 11 , wherein any number of experimental simulations, resulting from applying the mathematics described for claim 1 iteratively, may be compared in a graphic and/or tabular form within the system and favorable designs selected and retained.
13 . A method of designing a multi-arm multistage (MAMS) experiment, the method comprising:
receiving input parameters comprising experiment design parameters and experiment design boundaries;
computing boundaries;
computing sample size;
executing a first simulation of the MAMS experiment to determine a first set of boundaries efficacy;
modifying one or more input parameters;
executing a second simulation of the MAMS experiment to determine a second set of boundaries efficacy; and
comparing the first set of boundaries efficacy with the second set of boundaries efficacy to identify a preferred boundaries efficacy.
14 . A method for constructing a D-arm J-stage design for N number of stages, the method comprising:
computing a first boundary for a first stage N using a distribution of D-score statistics at the first stage;
incrementing a stage value N by one and advancing to a next stage;
updating the distribution of D-score statistics at a second stage for a second stage N+1;
computing a second boundary for the second stage N+1 using the distribution of D-score statistics at a second stage;
incrementing a stage value N+1 by one and advancing to a next stage; and
continuing incrementing stage values, advancing stages, and updating the distribution of D-score statistics to correspond to the stage values until a stage value is equal to J number of stages.