IP Library › Granted Patent US 12,211,593
Granted Patent B2
US 12,211,593 · App. 17/163,427 · Granted Jan 28, 2025

Trial design platform with recommendation engine

Inventors: Jaydeep Bhattacharyya (Acton, MA); James Bolognese (Woodbridge, NJ); Alexandre Buer (Holliston, MA); Eric Edwards (Nolensville, TN); Stanley Y. Huang (Wellesley, MA); Yannis Jemiai (Lexington, MA); Cyrus Mehta (Cambridge, MA); Nitin Patel (Cambridge, MA); Anne Pelz (Arlington, MA); Ajay Prabhakar Sathe (Pune, IN); Joshua A. Schultz (Boston, MA); Pralay Senchaudhuri (Cambridge, MA)
Assignee: Cytel Inc.
G16H10/20G06F30/10G06F30/12G06F30/20G06N5/04G06N20/00G06Q10/06315G06Q10/067G06Q30/0205G16H40/20G16H50/70G06F2111/02G06F2111/04G06F2111/06G06F2111/08G06F2111/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,211,593
App. No.
17/163,427
Granted
Jan 28, 2025
Kind
B2
Abstract

A method, according to some implementations, includes obtaining trial design simulation results for a set of trial designs and determining a set of Pareto designs in the set of trial designs based at least in part on the trial design simulation results and one or more performance parameters. The method further includes determining a set of convex hull designs in the set of trial designs, determining a set of recommended designs based at least in part on the set of Pareto designs and the set of convex hull designs, and transmitting the set of recommended designs.

Claims (74)

1. A method comprising:

generating simulation parameters, wherein at least one parameter includes a placeholder value comprising an estimated value and/or a predicted value;

simulating trial designs based on the simulation parameters;

obtaining trial design simulation results for a set of trial designs;

supplementing the placeholder value of the trial design simulation results with substitute data comprising historical data and/or real-world data;

determining a set of Pareto designs in the set of trial designs based at least in part on the trial design simulation results and one or more performance parameters;

determining a set of convex hull designs in the set of trial designs;

determining a set of recommended designs based at least in part on the set of Pareto designs and the set of convex hull designs; and

transmitting the set of recommended designs.

2. The method of claim 1 further comprising:

filtering one or more of the set of trial designs that are dominated by at least one of the set of convex hull designs.

3. The method of claim 1 further comprising:

filtering one or more of the set of trial designs that are dominated by at least one of the set of Pareto designs.

4. The method of claim 1 , wherein determining the set of recommended designs comprises:

determining at least one of the set of recommended designs within an epsilon-distance from at least one of the set of Pareto designs.

5. The method of claim 1 , wherein determining the set of recommended designs comprises:

determining at least one of the set of recommended designs within an epsilon-distance from at least one of the set of convex hull designs.

6. The method of claim 1 , wherein determining a set of recommended designs comprises:

performing simulated annealing on one or more of the set of trial designs.

7. The method of claim 6 , wherein simulated annealing is based at least in part on facets of the set of convex hull designs.

8. The method of claim 1 further comprising:

identifying different design types in the set of Pareto designs.

9. The method of claim 1 , wherein:

the set of Pareto designs is determined prior to the set of convex hull designs;

the set of convex hull designs is derived from the set of Pareto designs such that each of the set of convex hull designs is in the set of Pareto designs; and

at least one of the set of recommended designs is in the set of convex hull designs.

10. The method of claim 1 , wherein:

the set of convex hull designs is determined prior to the set of Pareto designs;

the set of Pareto designs is derived from the set of convex hull designs such that each of the set of Pareto designs is in the set of convex hull designs; and

at least one of the set of recommended designs is in the set of convex hull designs.

11. The method of claim 1 further comprising:

identifying a number of trial designs in the set of Pareto designs;

wherein:

determining the set of convex hull designs occurs when the number is greater-than-or-equal-to a threshold; and

the set of convex hull designs is derived from the set of Pareto designs.

12. The method of claim 1 further comprising:

identifying a number of trial designs in the set of Pareto designs; and

if the number is less-than-or-equal-to a threshold, identifying, for inclusion in the set of recommended designs, one or more of the set of trial designs within an epsilon distance of at least one of the set of Pareto designs.

13. The method of claim 1 further comprising:

for each of the set of Pareto designs, determining a design type;

determining a first trial design of the set of Pareto designs is of a different design type from and within an epsilon-distance to a second trial design of the set of Pareto designs; and

including the first trial design and the second trial design in the set of recommended designs.

14. The method of claim 1 , further comprising:

evaluating historical trial design selections to identify one or more trial design parameters based at least in part on one or more trial design criteria determined from a user via an interactive interface, wherein obtaining trial design simulation results is based at least in part on a quick search data structure and the one or more trial design parameters;

generating a substitute for at least some of the trial design simulation results based at least in part on a relationship between the trial design simulation results and supplemental data;

generating a performance surface based at least in part on a set of trial designs corresponding to the trial design simulation results;

evaluating one or more trial designs based at least in part on the performance surface; and

calculating a score based on normalized score component values corresponding to the trial design simulation results.

15. An apparatus comprising:

an evaluation circuit structured to generate simulation parameters, wherein at least one parameter includes a placeholder value comprising an estimated value and/or a predicted value;

a simulation circuit structured to simulate trial designs based on the simulation parameters;

a results processing circuit structured to interpret trial design simulation results for a set of trial designs;

a substitute circuit structured to supplement the placeholder value of the trial design simulation results with substitute data comprising historical data and/or real-world data;

a Pareto evaluation circuit structured to determine a set of Pareto designs in the set of trial designs based at least in part on the trial design simulation results and one or more performance parameters;

a convex hull evaluation circuit structured to determine a set of convex hull designs in the sct of trial designs;

a recommendation circuit structured to determine a set of recommended designs based at least in part on the set of Pareto designs and the set of convex hull designs; and

a recommendation provisioning circuit structured to transmit the set of recommended designs.

16. The apparatus of claim 15 further comprising:

a filtering circuit structured to filter one or more of the set of trial designs that are dominated by at least one of the set of convex hull designs.

17. The apparatus of claim 15 further comprising:

a filtering circuit structured to filter one or more of the set of trial designs that are dominated by at least one of the set of Pareto designs.

18. The apparatus of claim 15 , wherein the recommendation circuit is further structured to:

determine at least one of the set of recommended designs within an epsilon-distance from at least one of the set of Pareto designs.

19. A non-transitory computer-readable medium storing instructions that adapt at least one processor to:

generate simulation parameters, wherein at least one parameter includes a placeholder value comprising an estimated value and/or a predicted value;

simulate trial designs based on the simulation parameters;

interpret trial design simulation results for a set of trial designs;

supplement the placeholder value of the trial design simulation results with substitute data comprising historical data and/or real-world data;

determine a set of Pareto designs in the set of trial designs based at least in part on the trial design simulation results and one or more performance parameters;

determine a set of convex hull designs in the set of trial designs;

determine a set of recommended designs based at least in part on the set of Pareto designs and the set of convex hull designs; and

transmit the set of recommended designs.

20. The non-transitory computer-readable medium of claim 19 , wherein the stored instructions further adapt the at least one processor to:

filter one or more of the set of trial designs that are dominated by at least one of the set of convex hull designs.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 10, 2021
From: BHATTACHARYYA, JAYDEEP; BOLOGNESE, JAMES; BUER, ALEXANDRE; EDWARDS, ERIC; HUANG, STANLEY Y.; JEMIAI, YANNIS; MEHTA, CYRUS; PATEL, NITIN; PELZ, ANNE; SATHE, AJAY PRABHAKAR; SCHULTZ, JOSHUA A.; SENCHAUDHURI, PRALAY
To: CYTEL INC.
Reel/Frame 056186/0679 →
Continuity (7)
Provisional Application 62968874 · Jan 31, 2020
Provisional Application 63002197 · Mar 30, 2020
Provisional Application 63002253 · Mar 30, 2020
Provisional Application 63037977 · Jun 11, 2020
Provisional Application 63085700 · Sep 30, 2020
Provisional Application 63086474 · Oct 1, 2020
Related Publication 20210241144A1 · Aug 5, 2021
References Cited (123)
US 8781882B1 · Arboletti et al. · 2014 [cited by applicant]
US 10010291B2 · Budiman et al. · 2018 [cited by applicant]
US 10878169B2 · Goel et al. · 2020 [cited by applicant]
US 11733687B2 · Stump et al. · 2023 [cited by applicant]
US 12040059B2 · Bhattacharyya et al. · 2024 [cited by applicant]
US 20020077853A1 · Boru et al. · 2002 [cited by applicant]
US 20030065669A1 · Kahn et al. · 2003 [cited by applicant]
US 20030088320A1 · Sale · 2003 [cited by examiner]
US 20050222862A1 · Guhde et al. · 2005 [cited by applicant]
US 20060041403A1 · Jaber · 2006 [cited by applicant]
US 20070026365A1 · Friedrich et al. · 2007 [cited by applicant]
US 20070292012A1 · Brandon · 2007 [cited by examiner]
US 20080010044A1 · Ruetsch · 2008 [cited by examiner]
US 20080133270A1 · Michelson et al. · 2008 [cited by applicant]
US 20080215512A1 · Narzisi et al. · 2008 [cited by applicant]
US 20080256006A1 · Buscema · 2008 [cited by examiner]
US 20080312951A1 · Herpichboehm et al. · 2008 [cited by applicant]
US 20080313025A1 · Chowdhury · 2008 [cited by applicant]
US 20090292554A1 · Schultz · 2009 [cited by examiner]
US 20100211411A1 · Hudson · 2010 [cited by applicant]
US 20100268544A1 · Nitahara et al. · 2010 [cited by applicant]
US 20110301982A1 · Green, Jr. · 2011 [cited by examiner]
US 20120089418A1 · Kamath et al. · 2012 [cited by applicant]
US 20120154511A1 · Hsu et al. · 2012 [cited by applicant]
US 20130132042A1 · Chan · 2013 [cited by applicant]
US 20130197878A1 · Fiege et al. · 2013 [cited by applicant]
US 20130304504A1 · Powell · 2013 [cited by examiner]
US 20130304542A1 · Powell · 2013 [cited by applicant]
US 20130346101A1 · Kahn et al. · 2013 [cited by applicant]
US 20140006039A1 · Khan et al. · 2014 [cited by applicant]
US 20140006042A1 · Keefe et al. · 2014 [cited by applicant]
US 20140122126A1 · Riskin · 2014 [cited by applicant]
US 20140350966A1 · Khatana et al. · 2014 [cited by applicant]
US 20150073830A1 · Hill et al. · 2015 [cited by applicant]
US 20150088783A1 · Mun · 2015 [cited by applicant]
US 20150105878A1 · Jones et al. · 2015 [cited by applicant]
US 20150106110A1 · Edwards et al. · 2015 [cited by applicant]
US 20160129282A1 · Yin et al. · 2016 [cited by applicant]
US 20160198223A1 · Maluk et al. · 2016 [cited by applicant]
US 20160239619A1 · Abou-Hawili et al. · 2016 [cited by applicant]
US 20160255139A1 · Rathod · 2016 [cited by applicant]
US 20160314280A1 · Fusari · 2016 [cited by examiner]
US 20170147794A1 · Harder · 2017 [cited by examiner]
US 20170213307A1 · Platt et al. · 2017 [cited by applicant]
US 20170329880A1 · Ghosh et al. · 2017 [cited by applicant]
US 20180046780A1 · Graiver · 2018 [cited by examiner]
US 20180063386A1 · Sharma et al. · 2018 [cited by applicant]
US 20180165808A1 · Bagci et al. · 2018 [cited by applicant]
US 20180239524A1 · Snibbe et al. · 2018 [cited by applicant]
US 20190006024A1 · Kapoor · 2019 [cited by examiner]
US 20190019570A1 · Fuertinger · 2019 [cited by examiner]
US 20190103192A1 · Bent et al. · 2019 [cited by applicant]
US 20190259291A1 · Pradhan et al. · 2019 [cited by applicant]
US 20190261908A1 · Alailima et al. · 2019 [cited by applicant]
US 20190355459A1 · Li et al. · 2019 [cited by applicant]
US 20190392075A1 · Han et al. · 2019 [cited by applicant]
US 20200034690A1 · Ghose et al. · 2020 [cited by applicant]
US 20200075148A1 · Nguyen et al. · 2020 [cited by applicant]
US 20200090796A1 · Krishnan · 2020 [cited by examiner]
US 20200105380A1 · Ennist · 2020 [cited by examiner]
US 20200176098A1 · Lucas et al. · 2020 [cited by applicant]
US 20200258599A1 · Clark · 2020 [cited by examiner]
US 20200258600A1 · Will · 2020 [cited by examiner]
US 20200286596A1 · Yang et al. · 2020 [cited by applicant]
US 20200303075A1 · Krishna et al. · 2020 [cited by applicant]
US 20200321083A1 · Vergetis · 2020 [cited by examiner]
US 20200365239A1 · Sabharwal · 2020 [cited by examiner]
US 20200411141A1 · Foster et al. · 2020 [cited by applicant]
US 20200411199A1 · Shrager et al. · 2020 [cited by applicant]
US 20210090694A1 · Colley et al. · 2021 [cited by applicant]
US 20210097444A1 · Bansal et al. · 2021 [cited by applicant]
US 20210110075A1 · Dalloro et al. · 2021 [cited by applicant]
US 20210158906A1 · Xie · 2021 [cited by examiner]
US 20210166330A1 · Baker et al. · 2021 [cited by applicant]
US 20210235008A1 · Su et al. · 2021 [cited by applicant]
US 20210240883A1 · Bhattacharya et al. · 2021 [cited by applicant]
US 20210240884A1 · Bhattacharya et al. · 2021 [cited by applicant]
US 20210240885A1 · Bhattacharya et al. · 2021 [cited by applicant]
US 20210240886A1 · Bhattacharya et al. · 2021 [cited by applicant]
US 20210241144A1 · Bhattacharya · 2021 [cited by examiner]
US 20210241859A1 · Bhattacharya et al. · 2021 [cited by applicant]
US 20210241860A1 · Bhattacharya et al. · 2021 [cited by applicant]
US 20210241861A1 · Bhattacharya et al. · 2021 [cited by applicant]
US 20210241862A1 · Bhattacharya et al. · 2021 [cited by applicant]
US 20210241863A1 · Bhattacharya · 2021 [cited by examiner]
US 20210241864A1 · Bhattacharya et al. · 2021 [cited by applicant]
US 20210241865A1 · Bhattacharya · 2021 [cited by examiner]
US 20210241866A1 · Bhattacharya · 2021 [cited by examiner]
US 20210257061A1 · Li · 2021 [cited by examiner]
US 20210319158A1 · Bhattacharyya · 2021 [cited by examiner]
US 20210378747A1 · Emili et al. · 2021 [cited by applicant]
US 20220374558A1 · Patel et al. · 2022 [cited by applicant]
US 20220375551A1 · Patel et al. · 2022 [cited by applicant]
US 20220382935A1 · Patel et al. · 2022 [cited by applicant]
WO 0217211A2 · 2002 [cited by applicant]
WO 2010043240A1 · 2010 [cited by applicant]
WO 2016024915A1 · 2016 [cited by applicant]
WO 2016170368A1 · 2016 [cited by applicant]
WO 2019045637A2 · 2019 [cited by applicant]
WO 2019144116A1 · 2019 [cited by applicant]
WO 2020188341A1 · 2020 [cited by applicant]
WO 2021155329A1 · 2021 [cited by applicant]
WO 2022271876A1 · 2022 [cited by applicant]
Deb et al., “Evaluating the ε-Domination Based Multi-Objective Evolutionary Algorithm for a Quick Computation of Pareto-Optimal Solutions”, 2005, Massachusetts Institute of Technology, Evolutionary Computation 13(4), pp… [cited by examiner]
Lee, Kim May, et al., “Design of experiments for a confirmatory trial of precision”, In: Journal of Statistical Planning and Inference 199 (2019) 179-187, Jun. 23, 2018, [online] [retrieved on May 12, 2021 (May 12, 2021… [cited by applicant]
PCT/US2021/015954 , “International Application Serial No. PCT/US2021/015954, International Search Report and Written Opinion mailed Jun. 3, 2021”, PCT/US2021/015954, 13 pages. [cited by applicant]
PCT/US2021/015954 , “International Application Serial No. PCT/US2021/015954, Invitation to Pay Additional Fees and, Where Applicable, Protest Fee mailed Apr. 1, 2021”, Cytel Inc., 3 pages. [cited by applicant]
21748352.8 , “European Application Serial No. 21748352.8, Extended European Search Report mailed Jan. 31, 2024”, Cytel Inc., 7 pages. [cited by applicant]
Alastair, Aitchison , “In Defence of 3D Charts”, Apr. 29, 2011, 6 pages. [cited by applicant]
Chen, Qi , et al., “An optimization framework to combine operable space maximization with design of experiments.”, AlChE Journal., 2018, 14 pages. [cited by applicant]
Grayling , et al., “Admissible multiarm stepped-wedge cluster randomized trial designs”, John Wiley & Sons Ltd, 2017, pp. 1103-1119. [cited by applicant]
Huntington, Nick Klein, “Robustness Tests: What, Why, and How?”, NickCHK, retrieved via Wayback Machine, all pages, https://web.archive.org/web/20191123233011/https://www.nickchk.com/robustness.html, 2019, 7 pages. [cited by applicant]
Ismail, Abbas , “Optimal design of clinical trials with computer simulation based on results of earlier trials, illustrated with a lipodystrophy trial in HIV patients”, Apr. 29, 2008, 9 pages. [cited by applicant]
Lawrence, M. Friedman, et al., “Fundamentals of Clinical Trials”, 4th edition, , Springer, 2010, 464 pages. [cited by applicant]
Mozaffari, Ahmad , et al., “An introduction to synchronous self-learning Pareto strategy”, 2013, arXiv.org, Cornell University, all pages, all figures, all tables, https://arxiv.org/abs/1312.4132, 2013, 17 pages. [cited by applicant]
PCT/US2021/015954 , “International Application Serial No. PCT/US2021/015954, International Preliminary Report on Patentability mailed Aug. 11, 2022”, Cytel Inc., 11 pages. [cited by applicant]
PCT/US2022/034599 , “International Application Serial No. PCT/US2022/034599, International Search Report and Written Opinion mailed Dec. 5, 2022”, Cytel Inc., 17 pages. [cited by applicant]
PCT/US2022/034599 , “International Application Serial No. PCT/US2022/034599, Invitation to Pay Additional Fees and, Where Applicable, Protest Fee mailed Sep. 28, 2022”, Cytel Inc., 3 pages. [cited by applicant]
PCT/US2022/034599 , “International Application Serial No. PCT/US2022/034599, International Preliminary Search Report on Patentability and Written Opinion mailed Dec. 14, 2023”, Cytel Inc., 9 pages. [cited by applicant]
Qi , et al., “A Delaunay Triangulation Based Density Measurement for Evolutionary Multi-objective Optimization”, Springer Artificial Life and Computational Intelligence, Second Australasian Conference ACALCI 2016 Procee… [cited by applicant]
Sverdlov , “On Optimal Designs for Clinical Trials: An Updated Review”, 2019, pp. 1-29. [cited by applicant]
Wikipedia , “Robust statistics—Wikipedia”, retrieved via Wayback Machine, all pages, https://web.archive.org/web/20191224214056/https://en.wikipedia.org/wiki/Robust_statistics, 2019, 16 pages. [cited by applicant]
Zhao , “Systematic data-driven modeling . . . experimental design and hypothesis evaluation”, Available from ProQuest Dissertations and Theses Professional. (304875988). Retrieved from https://dialog.proquest.com/profes… [cited by applicant]