TRIAL DESIGN BENCHMARKING PLATFORM

1 . A method comprising:

providing inputs to a plurality of trial design simulation engines;

receiving first outputs of the plurality of trial design simulation engines in response to the inputs;

providing variations of the inputs to the plurality of trial design simulation engines;

receiving second outputs of the plurality of trial design simulation engines in response to the variations;

evaluating the first and the second outputs to determine delta values; and

determining, based in part on the delta values, a plurality of normalization factors for the plurality of trial design simulation engines.

2 . The method of claim 1 , wherein a first trial design simulation engine of the plurality is structured to simulate a first trial design that is of a different type than a second trial design which a second trial design simulation engine of the plurality is structured to simulate.

3 . The method of claim 2 , wherein a first trial design simulation engine of the plurality is a different version of a second trial design simulation engine of the plurality.

4 . The method of claim 2 , wherein a first trial design simulation engine of the plurality was generated by a first entity and a second trial design simulation engine of the plurality was generated by a second entity of the plurality distinct from the first entity.

5 . The method of claim 2 , wherein the first and the second outputs include metadata.

6 . The method of claim 2 , further comprising:

multiplying at least one of the first or the second outputs by at least one of the plurality of normalization factors.

7 . The method of claim 2 , wherein each of the plurality of normalization factors differ in value based at least in part on differing performance criteria defined, in part, by the inputs.

8 . The method of claim 2 , further comprising:

determining, for each of the trial design simulation engines, an output variability; and

displaying the variabilities to a user.

9 . The method of claim 2 , further comprising:

determining a valid range of output values based on the delta values; and

determining that at least one of the plurality of trial design simulation engines outputs invalid values.

10 . The method of claim 2 , further comprising:

evaluating historical trial design selections to identify one or more trial design parameters based at least in part on one or more trial design criteria determined from a user via an interactive interface, wherein receiving the first outputs is based at least in part on a quick search data structure and the one or more trial design parameters;

generating a substitute for at least some of the first outputs based at least in part on a relationship between the first outputs and supplemental data;

generating a performance surface based at least in part on a set of trial designs corresponding to first outputs;

evaluating one or more trial designs based at least in part on the performance surface; and

calculating a score based on normalized score component values corresponding to the first outputs.

11 . A method comprising:

providing inputs to a plurality of trial design simulation engines;

receiving outputs of the plurality of trial design simulation engines in response to the inputs;

comparing outputs to expected outputs; and

determining, based on the comparing, a plurality of normalization factors for the plurality of trial design simulation engines.

12 . The method of claim 11 , wherein a first trial design simulation engine of the plurality is structured to simulate a first trial design that is of a different type than a second trial design which a second trial design simulation engine of the plurality is structured to simulate.

13 . The method of claim 11 , wherein a first trial design simulation engine of the plurality is a different version of a second trial design simulation engine of the plurality.

14 . The method of claim 11 , wherein a first trial design simulation engine of the plurality was generated by a first entity and a second trial design simulation engine of the plurality was generated by a second entity of the plurality distinct from the first entity.

15 . The method of claim 11 , wherein each of the plurality of normalization factors differ in value based at least in part on differing performance criteria defined, in part, by the inputs.

16 . An apparatus comprising:

an output processing circuit structured to interpret output data from a plurality of trial design simulation engines;

a comparison circuit structured to compare the interpreted output data to expected output data;

a normalization circuit structured to determine a plurality of normalization factors for the plurality of trial design simulation engines; and

a normalization provisioning circuit structured to transmit the plurality of normalization factors.

17 . The apparatus of claim 16 , wherein a first trial design simulation engine of the plurality is structured to simulate a first trial design that is of a different type than a second trial design which a second trial design simulation engine of the plurality is structured to simulate.

18 . The apparatus of claim 16 , wherein a first trial design simulation engine of the plurality is a different version of a second trial design simulation engine of the plurality.

19 . The apparatus of claim 16 , wherein a first trial design simulation engine of the plurality was generated by a first entity and a second trial design simulation engine of the plurality was generated by a second entity of the plurality distinct from the first entity.

20 . The apparatus of claim 16 , wherein each of the plurality of normalization factors differ in value based at least in part on differing performance criteria.