Computer system and method for training, certifying or monitoring human clinical raters
A method and system are disclosed for computerized training, monitoring, certification or re-certification of human raters in clinical trials. The invention provides an interactive computerized interview that can be compared scores obtained by human raters to monitor and train clinical raters on a continuing basis in order to reduce rater drift and variance during clinical trials. Remediation can be automatically flagged or provided to improve a deviating rater's performance. Computerized assessment of symptom severity is provided without the need for human clinical raters. The system and method enable screening of prospective patients for inclusion or exclusion from a clinical trial by automatically obtaining computerized ratings of symptom severity.
1. A method comprising:
receiving one or more rater inputs reflecting the rater's clinical evaluation of a severity of a previously diagnosed condition in one or more subjects;
receiving one or more severity scores for the previously diagnosed condition in the one or more subjects, wherein said severity scores have been calculated by:
(a) presenting a first question to the subject and receiving a first input from the subject in response thereto;
(b) based on the first input, selecting a second question from a plurality of questions;
(c) presenting the selected second questions to the subject and receiving a second input from the subject in response thereto; and
(d) based on one or more inputs received from the subject, determining in accordance with a clinical rating scale the severity score for a previously diagnosed condition in the subject; and
determining, via a processor, whether the one or more rater inputs are in accordance with the one or more severity scores, respectively.
2. The method of claim 1 , further comprising receiving the one or more rater inputs via a keyboard, a numerical keypad, a dial, a touch screen, a touch pad, a pointing device, a microphone, a telephone, a joystick, a game pad, a satellite dish, fax, a scanner, a personal computer, a hand-held device, a multi-processor system, a network computer, or a minicomputer.
3. The method of claim 1 , further comprising receiving the one or more rater inputs via at least one of a universal serial bus (USB) or CD ROM.
4. The method of claim 1 , further comprising receiving the one or more rater inputs via a network.
5. The method of claim 4 , wherein the network comprises at least one of a local area network (LAN), a wide area network (WAN), or a wireless LAN (WLAN).
6. The method of claim 1 , further comprising receiving the one or more rater inputs via a webpage.
7. The method of claim 1 , wherein the step of determining whether the one or more rater inputs are in accordance with the one or more severity scores includes using a personal computer, a hand-held device, a multi-processor system, a network computer, a minicomputer, or a mainframe computer.
8. The method of claim 1 , further including one or more of the following steps:
(a) recertifying the rater;
(b) indicating that remedial measures are needed;
(c) providing remedial measures;
(d) eliminating the rater input for the purpose of analysis in the clinical trial; and
(e) determining whether to include or exclude the one or more subjects in a clinical trial.
9. The method of claim 1 , further including one or more of the following steps:
(a) indicating that remedial measures are needed;
(b) providing remedial measures;
(c) eliminating the rater input for the purpose of analysis in the clinical trial; and
(e) determining whether to include or exclude the one or more subjects in a clinical trial.
10. The method of claim 1 , further including a step of automatically determining by previously established criteria whether or not the one or more subjects qualifies for inclusion in the clinical trial.
11. A method for monitoring performance of a clinical rater participating in a clinical trial comprising:
receiving a rater input reflecting the rater's clinical evaluation of a severity of a previously diagnosed condition in a subject;
receiving a calculated severity score for the previously diagnosed condition in the subject, wherein said severity scores have been calculated by:
(a) presenting a first question to the subject and receiving a first input from the subject in response thereto;
(b) based on the first input, selecting a second question from a plurality of questions;
(c) presenting the selected second questions to the subject and receiving a second input from the subject in response thereto; and
(d) based on one or more inputs received from the subject, determining in accordance with a clinical rating scale the calculated severity score for a previously diagnosed condition in the subject; and
comparing, via a processor, the rater input with the calculated severity score, including determining if the rater input deviates from the calculated severity score by more than a given threshold.
12. The method of claim 11 , further comprising receiving the rater input via a keyboard, a numerical keypad, a dial, a touch screen, a touch pad, a pointing device, a microphone, a telephone, a joystick, a game pad, a satellite dish, fax, a scanner, a personal computer, a hand-held device, a multi-processor system, a network computer, or a minicomputer.
13. The method of claim 11 , further comprising receiving the rater input via a at least one of a universal serial bus (USB) or CD ROM.
14. The method of claim 11 , further comprising receiving the rater input via a network.
15. The method of claim 14 , wherein the network comprises at least one of a local area network (LAN), a wide area network (WAN), or a wireless LAN (WLAN).
16. The method of claim 11 , further comprising receiving the rater input via a webpage.
17. The method of claim 11 , wherein the step of determining whether the rater input is in accordance with the calculated severity score includes using a personal computer, a hand-held device, a multi-processor system, a network computer, a minicomputer, or a mainframe computer.
18. The method of claim 11 , further comprising one or more of the following steps:
(a) indicating that remedial measures are needed;
(b) providing remedial measures;
(c) eliminating the rater input for the purpose of analysis in the clinical trial; and
(d) determining whether to include or exclude the subjects in the clinical trial.
19. The method of claim 11 , further including a step of automatically determining by previously established criteria whether or not the one or more subjects qualifies for inclusion in the clinical trial.