Molecular evidence platform for auditable, continuous optimization of variant interpretation in genetic and genomic testing and analysis
Disclosed herein are system, method, and computer program product embodiments for optimizing the determination of a phenotypic impact of a molecular variant identified in molecular tests, samples, or reports of subjects by way of regularly incorporating, updating, monitoring, validating, selecting, and auditing the best-performing evidence models for the interpretation of molecular variants across a plurality of evidence classes.
1. A computer implemented method for predicting a phenotypic impact of a molecular variant of interest, the method comprising:
(a) recording an evidence model comprising evidence data, wherein the evidence data comprises objects, algorithms, and/or functions that yield predictions of phenotypic impacts of molecular variants for a target entity;
(b) determining validation performance data for the evidence model based on production data, wherein the production data represents a first plurality of molecular variants with associated phenotypic impacts derived from clinical data and/or population data, and wherein the validation performance data corresponds to a uniform set of performance metrics computed using the production data;
(c) determining test performance data for the evidence model based on the evidence data and test data in response to receiving the test data for the evidence model, wherein the test data comprises a second plurality of molecular variants with associated phenotypic impacts derived from clinical data and/or population data, wherein the second plurality of molecular variants are disjoint from the first plurality of molecular variants, and wherein the test performance data corresponds to the uniform set of performance metrics computed using the test data;
(d) ranking the evidence model in a set of evidence models for the target entity based on the validation performance data and/or the test performance data; and
(e) providing the predicted phenotypic impact using a best-performing evidence model for the target entity based on the ranking in response to a query for the predicted phenotypic impact of the molecular variant of interest for the target entity from a variant interpretation terminal.
2. The method of claim 1 , wherein the target entity comprises a functional element, molecule or molecular variant, and a phenotype of interest.
3. The method of claim 1 , wherein the recording of the evidence model comprises generating the evidence model based on the production data.
4. The method of claim 1 , further comprising generating a hash value of supporting data for the evidence model, wherein the supporting data is generated from the evidence data, the production data, the test data, the validation performance data, the test performance data, or a combination thereof.
5. The method of claim 1 , wherein the production data is received from a clinical knowledgebase.
6. The method of claim 1 , wherein the determining the validation performance data comprises:
(1) calculating a phenotype impact score for one or more molecular variants of the target entity in the production data using the evidence model and a model validation technique; and,
(2) generating the validation performance data based on the phenotype impact scores using the uniform set of performance metrics.
7. The method of claim 1 , wherein the determining the test performance data comprises:
(1) calculating a phenotype impact score for one or more molecular variants of the target entity in the test data using the evidence model and a model validation technique; and,
(2) generating the test performance data based on the phenotype impact scores using a the uniform set of performance metrics.
8. The method of claim 3 , wherein the generating the evidence model based on the production data uses:
(i) a machine learning technique;
(ii) a functional assay;
(iii) a biophysical simulation; or,
(iv) a combination thereof.
9. The method of claim 8 , wherein the machine learning technique is unsupervised, supervised, or semi-supervised.
10. The method of claim 4 , further comprising: providing an auditing record to the variant interpretation terminal, wherein:
(i) the auditing record references an entry for the supporting data in the database, and
(ii) the auditing record enables the variant interpretation terminal to audit content of the supporting data and a time of creation of the supporting data.
11. The method of claim 1 , wherein the uniform set of performance metrics comprises one or more diagnostic metrics, classification metrics, or regression accuracy metrics.
12. The method of claim 11 , wherein the diagnostic metrics comprises one or more of the following: raw accuracy, balanced accuracy, true positive rate, true negative rate, positive predictive value, negative predictive value, true positive, true negative, false positive, false negative, and coverage.
13. A system for predicting a phenotypic impact of a molecular variant of interest, the system comprising:
(i) a memory having computer-readable instructions stored thereon; and
(ii) at least one processor coupled to the memory;
wherein the computer-readable instructions, when executed by the at least one processor, cause the at least one processor to:
(a) record an evidence model comprising evidence data, wherein the evidence data comprises objects, algorithms, and/or functions that yield predictions of phenotypic impacts of molecular variants for a target entity;
(b) determine validation performance data for the evidence model based on production data, wherein the production data represents a first plurality of molecular variants with associated phenotypic impacts derived from clinical data and/or population data, and wherein the validation performance data corresponds to a uniform set of performance metrics computed using the production data;
(c) determine test performance data for the evidence model based on the evidence data and test data in response to receiving the test data for the evidence model, wherein the test data comprises a second plurality of molecular variants with associated phenotypic impacts derived from clinical data and/or population data, wherein the second plurality of molecular variants are disjoint from the first plurality of molecular variants, and wherein the test performance data corresponds to the uniform set of performance metrics computed using the test data;
(d) rank the evidence model in a set of evidence models for the target entity based on the validation performance data and/or the test performance data; and
(e) provide the predicted phenotypic impact using a best-performing evidence model for the target entity based on the ranking in response to a query for the predicted phenotypic impact of the molecular variant of interest for the target entity from a variant interpretation terminal.
14. The system of claim 13 , wherein the target entity comprises a functional element, molecule or molecular variant, and a phenotype of interest.
15. The system of claim 13 , wherein the computer-readable instructions that cause the at least one processor to record the evidence model comprise computer-readable instructions that cause the at least one processor to generate the evidence model based on the production data.
16. The system of claim 13 , wherein the computer-readable instructions, when executed by the at least one processor, further cause the at least one processor to generate a hash value of supporting data for the evidence model, wherein the supporting data is generated from the evidence data, the production data, the test data, the validation performance data, the test performance data, or a combination thereof.
17. The system of claim 13 , wherein the computer-readable instructions that cause the at least one processor to determine the validation performance data comprise computer-readable instructions that cause the at least one processor to:
(1) calculate a phenotype impact score for one or more molecular variants of the target entity in the production data using the evidence model and a model validation technique; and,
(2) generate the validation performance data based on the phenotype impact scores using the uniform set of performance metrics.
18. The system of claim 13 , wherein the computer-readable instructions that cause the at least one processor to determine the test performance data comprise computer-readable instructions that cause the at least one processor to:
(1) calculate a phenotype impact score for one or more molecular variants of the target entity in the test data using the evidence model and a model validation technique; and,
(2) generate the test performance data based on the phenotype impact scores using the uniform set of performance metrics.
19. The system of claim 13 , wherein the uniform set of performance metrics comprises one or more diagnostic metrics, classification metrics, or regression accuracy metrics.
20. The system of claim 19 , wherein the diagnostic metrics comprise one or more of the following: raw accuracy, balanced accuracy, true positive rate, true negative rate, positive predictive value, negative predictive value, true positive, true negative, false positive, false negative, and coverage.