METHODS FOR IMPROVING TEST EFFICIENCY AND ACCURACY IN A COMPUTER ADAPTIVE TEST (CAT)
A method for use of pretest items in a test to calculate interim scores is provided. The method includes, for example, a computer implemented test having a plurality of test items that include, for example, a plurality of operational items and one or more pretest 5 items having one or more pretest item parameters. An interim latent construct estimate is calculated using both operational and pretest items. The error for the latent construct estimation is controlled by weighting the contribution of the one or more pretest items.
1 . A method for generating interim scores with a computer adaptive testing platform by using examinee responses to pretest items to improve scoring accuracy, the method comprising:
obtaining a computer adaptive test from a test script administration module, the computer adaptive test comprising a plurality of operational items and one or more pretest items corresponding to one or more pretest items parameters, wherein the pretest item parameters include statistical pretest item response data acquired from other examinees;
calculating, with a latent construct estimator, an examinee interim score based on the one or more pretest item parameters and examinee responses to both operational items and pretest items;
storing the examinee interim score in a datastore; and
re-defining pretest item parameters for one or more of the pretest items based on the examinee responses during the administration of the computer adaptive test to control for error.
2 . The method of claim 1 further comprising displaying the computer adaptive test on a graphical user interface and obtaining the examinee responses from the graphical user interface.
3 . The method of claim 1 further comprising printing the computer adaptive test and obtaining the examinee responses from a scanner.
4 . The method of claim 1 further comprising updating the operational item parameters during administration of the computer adaptive test.
5 . The method of claim 1 further comprising using the computer adaptive testing module to set the one or more pretest item parameters to an average of a set of calibrated parameters for the plurality of operational items.
6 . The method of claim 1 further comprising updating the one or more pretest item parameters based on corresponding pretest item responses from a threshold number of additional examinees.
7 . The method of claim 1 wherein the pretest item parameters comprise maximum likelihood estimators based on the examinee responses.
8 . A system for generating interim scores during the administration of a computer adaptive test, the system comprising:
a computer adaptive testing server, a data store, and a graphical user interface, the computer adaptive testing server comprising a computer adaptive testing component and a test script administrator, the computer adaptive testing component configured to:
generate a computer adaptive test with the test script administrator, the computer adaptive test comprising a plurality of operational items and one or more pretest items having one or more pretest items parameters, wherein the pretest item parameters include statistical pretest item response data acquired from other examinees;
calculate an examinee interim score based on the one or more pretest item parameters and examinee responses to both operational items and pretest items; and
store the examinee interim score in a datastore; and
re-define pretest item parameters for one or more of the pretest items based on the examinee responses during the administration of the computer adaptive test to control for error.
9 . The system of claim 8 , wherein the computer adaptive testing component is further configured to display the computer adaptive test on the graphical user interface and obtain the examinee responses from the graphical user interface.
10 . The system of claim 8 , wherein the computer adaptive testing component is further configured to print the computer adaptive test on a printer and obtain the examinee responses from a scanner.
11 . The system of claim 8 , wherein the computer adaptive testing component is further configured to update the operational item parameters during administration of the computer adaptive test.
12 . The system of claim 8 , wherein the computer adaptive testing component is further configured to set the one or more pretest item parameters to an average of a set of calibrated parameters for the plurality of operational items.
13 . The system of claim 12 , wherein the computer adaptive testing component is further configured to update the one or more pretest item parameters based on corresponding pretest item responses from a threshold number of additional examinees.
14 . The system of claim 8 wherein the pretest item parameters comprise maximum likelihood estimators based on the examinee responses.