AI AND ML ASSISTED SYSTEM FOR DETERMINING SITE COMPLIANCE USING SITE VISIT REPORT
Methods and systems to automatically construct a clinical study site visit report (SVR), conduct the SVR, evaluate the SVR in real-time, and provide feedback while the SVR is being conducted. Responses to the SVR include user-selectable answers and natural language notes. Each response is evaluated as it is submitted based on a combination of pre-configured rules and a computer-trained model. If an anomaly is detected and is not already captured in the SVR, an alert is generated during performance of the SVR. The alert may include recommended remedial action.
1 . A machine-implemented method, comprising:
presenting of a site visit report in a sequential fashion on a user device during a site visit;
receiving responses to the questions via the user device, wherein the responses include user-selectable answers and natural language notes of a user;
evaluating each response as it is received from the user device to detect an anomaly in the clinical trial site visit, including evaluating the user-selected answers and the text analytics based on a combination of pre-configured rules and a computer-trained model, wherein the anomaly includes a protocol deviation and/or an adverse event, and wherein the text analytics includes sentiment analytics and/or topical analytics;
determining if the detected anomaly is already identified as an anomaly in the site visit report; and
generating an alert, during the site visit, if the detected anomaly is not already identified as an anomaly in the site visit report, wherein the alert includes a recommendation to resolve the anomaly.
2 . The method of claim 1 , further comprising:
selecting the questions to include in the site visit report based on features of a site and an associated clinical study;
configuring the rules to identify anomalies in the responses; and
training the model to correlate historical medical data with supervisor-identified anomalies in the historical medical data, wherein the historical medical data includes patient data, trial data, and laboratory test results.
3 . The method of claim 1 , wherein:
the evaluating comprises computing a compliance score for each response and detecting the anomaly when the compliance score exceeds a threshold.
4 . The method of claim 1 , wherein:
the generating an alert comprises ranking the detected anomaly based on a safety-related risk factor associated with the anomaly, during the site visit.
5 . The method of claim 1 , further comprising:
training a probabilistic topic model to detect topics from historical natural language notes associated with historical medical data; and
training a sentiment model to detect sentiments from the historical natural language notes;
wherein the evaluating comprises computing the text analytics with the probabilistic topic model and the sentiment model.
6 . The method of claim 1 , further comprising:
training the model to correlate text analytics extracted from historical natural language notes associated with historical medical data, and answers of historical site visit reports, with corresponding supervisor-declared adverse events;
wherein the evaluating comprises evaluating the text analytics and at least a subset of the responses with the trained model.
7 . The method of claim 1 , further comprising:
evaluating multiple site visit reports in combination with one another to detect a pattern of anomalies.
8 . A non-transitory computer readable medium encoded with a computer program that comprises instructions to cause a processor to:
present questions of a site visit report in a sequential fashion on a user device during a site visit;
receive responses to the questions via the user device, wherein the responses include user-selectable answers and natural language notes of a user;
evaluate each response as it is received from the user device to detect an anomaly in the site visit, including to evaluate the user-selected answers and text analytics of the natural language notes based on a combination of pre-configured rules and a computer-trained model, wherein the anomaly includes a protocol deviation and/or an adverse event, and wherein the text analytics includes sentiment analytics and/or topical analytics;
determine if the detected anomaly is already identified as an anomaly in the site visit report; and
generate an alert, during the site visit, if the detected anomaly is not already identified as an anomaly in the site visit report, wherein the alert includes a recommendation to resolve the anomaly.
9 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
select the questions to include in the site visit report based on features of a site and an associated clinical study;
configure the rules to identify anomalies in the responses; and
train the model to correlate historical medical data with supervisor-identified anomalies in the historical medical data, wherein the historical medical data includes patient data, trial data, and laboratory test results.
10 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
compute a compliance score for each of the responses; and
detect the anomaly when the compliance score exceeds a threshold.
11 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
rank the detected anomaly based on a safety-related risk factor associated with the anomaly, during the site visit.
12 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
train a probabilistic topic model to detect topics from historical natural language notes associated with historical medical data;
train a sentiment model to detect sentiments from the historical natural language notes; and
compute the text analytics with the probabilistic topic model and the sentiment model.
13 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
train the model to correlate text analytics extracted from historical natural language notes associated with historical medical data, and answers of historical site visit reports, with corresponding supervisor-declared adverse events; and
evaluate the text analytics and at lease a subset of the responses with the trained model.
14 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
evaluate multiple site visit reports in combination with one another to detect a pattern of deviations and/or anomalies.
15 . An apparatus, comprising a processor and memory configured to:
present questions of a site visit report in a sequential fashion on a user device during a site visit;
receive responses to the questions via the user device, wherein the responses include user-selectable answers and natural language notes of a user;
evaluate each response as it is received from the user device to detect an anomaly in the site visit, including to evaluate the user-selected answers and text analytics of the natural language notes based on a combination of pre-configured rules and a computer-trained model, wherein the anomaly includes a protocol deviation and/or an adverse event, and wherein the text analytics includes sentiment analytics and/or topical analytics;
determine if the detected anomaly is already identified as an anomaly in the site visit report; and
generate an alert, during the site visit, if the detected anomaly is not already identified as an anomaly in the site visit report, wherein the alert includes a recommendation to resolve the anomaly.
16 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
select the questions to include in the site visit report based on features of a site and an associated clinical study;
configure the rules to identify anomalies in the responses; and
train the model to correlate historical medical data with supervisor-identified anomalies in the historical medical data, wherein the historical medical data includes patient data, trial data, and laboratory test results.
17 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
compute a compliance score for each of the responses; and
detect the anomaly when the compliance score exceeds a threshold.
18 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
rank the detected anomaly based on a safety-related risk factor associated with the anomaly, during the site visit.
19 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
train a probabilistic topic model to detect topics from historical natural language notes associated with historical medical data;
train a sentiment model to detect sentiments from the historical natural language notes; and
compute the text analytics with the probabilistic topic model and the sentiment model.
20 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
train the model to correlate text analytics extracted from historical natural language notes associated with historical medical data, and answers of historical site visit reports, with corresponding supervisor-declared adverse events; and
evaluate the text analytics and at lease a subset of the responses with the trained model.
21 . The non-transitory computer readable medium of claim 8 , further including instructions to cause the processor to:
evaluate multiple site visit reports in combination with one another to detect a pattern of deviations and/or anomalies.