Systems and methods for detecting unusually frequent exactly matching and nearly matching test responses
Computer-implemented systems and methods are described herein for detecting unusually frequent exactly matching and nearly matching test responses. A plurality of test responses for each of a plurality of test takers is received. Evidence of a circulated key from the plurality of test responses is identified by detecting two or more test takers with exactly matching test responses for a plurality of questions. The strength of the evidence of the circulated key is analyzed by determining the probability that the two or more test takers would produce exactly matching test responses, wherein the strength of the evidence is inversely related to the probability. Test takers with nearly matching test responses to the circulated key are then identified. All test takers with exactly matching and nearly matching test responses are tagged for further investigation.
1. A computer-implemented method for detecting unusually frequent exactly matching and nearly matching test responses, comprising:
receiving a plurality of test responses for each of a plurality of test takers, wherein each test response corresponds to a question on a test distributed to the plurality of test takers;
identifying evidence of a circulated key from the plurality of test responses by detecting two or more test takers with exactly matching test responses for a plurality of questions, wherein the exactly matching test responses correspond to a response pattern from the circulated key;
analyzing a strength of the evidence of the circulated key by determining a probability that the two or more test takers would produce exactly matching test responses, wherein the strength of the evidence is inversely related to the probability;
identifying test takers with nearly matching test responses to the circulated key; and
tagging the test takers with exactly matching test responses and the test takers with nearly matching test responses for further investigation.
2. The computer-implemented method of claim 1 , wherein identifying test takers with nearly matching test responses to the circulated key comprises:
scoring the test responses of each test taker based on the circulated key;
scoring the test responses of each test taker based on a true key; and
comparing both scores, wherein a better score on the circulated key compared to the true key is evidence that the circulated key is being used.
3. The computer-implemented method of claim 1 , wherein evidence of more than one circulated key is identified and analyzed.
4. The computer-implemented method of claim 1 , wherein all of the test responses for each of a plurality of test takers are from a single testing center.
5. The computer-implemented method of claim 1 , wherein the plurality of test responses for each of a plurality of test takers are from more than one testing center.
6. A computer-implemented system for detecting unusually frequent exactly matching and nearly matching test responses, comprising:
one or more data processors;
one or more computer-readable storage mediums encoded with instructions for commanding the one or more data processors to execute steps that include:
receiving a plurality of test responses for each of a plurality of test takers, wherein each test response corresponds to a question on a test distributed to the plurality of test takers;
identifying evidence of a circulated key from the plurality of test responses by detecting two or more test takers with exactly matching test responses for a plurality of questions, wherein the exactly matching test responses correspond to a response pattern from the circulated key;
analyzing a strength of the evidence of the circulated key by determining a probability that the two or more test takers would produce exactly matching test responses, wherein the strength of the evidence is inversely related to the probability;
identifying test takers with nearly matching test responses to the circulated key; and
tagging the test takers with exactly matching test responses and the test takers with nearly matching test responses for further investigation.
7. The computer-implemented system of claim 6 , wherein identifying test takers with nearly matching test responses to the circulated key comprises:
scoring the test responses of each test taker based on the circulated key;
scoring the test responses of each test taker based on a true key; and
comparing both scores, wherein a better score on the circulated key compared to the true key is evidence that the circulated key is being used.
8. The computer-implemented system of claim 6 , wherein evidence of more than one circulated key is identified and analyzed.
9. The computer-implemented system of claim 6 , wherein all of the test responses for each of a plurality of test takers are from a single testing center.
10. The computer-implemented system of claim 6 , wherein the plurality of test responses for each of a plurality of test takers are from more than one testing center.
11. A non-transitory computer-readable storage medium comprising instructions for which when executed cause a processing system to execute steps comprising:
receiving a plurality of test responses for each of a plurality of test takers, wherein each test response corresponds to a question on a test distributed to the plurality of test takers;
identifying evidence of a circulated key from the plurality of test responses by detecting two or more test takers with exactly matching test responses for a plurality of questions, wherein the exactly matching test responses correspond to a response pattern from the circulated key;
analyzing a strength of the evidence of the circulated key by determining a probability that the two or more test takers would produce exactly matching test responses, wherein the strength of the evidence is inversely related to the probability;
identifying test takers with nearly matching test responses to the circulated key; and
tagging the test takers with exactly matching test responses and the test takers with nearly matching test responses for further investigation.
12. The non-transitory computer-readable storage medium of claim 11 , wherein identifying test takers with nearly matching test responses to the circulated key comprises:
scoring the test responses of each test taker based on the circulated key;
scoring the test responses of each test taker based on a true key; and
comparing both scores, wherein a better score on the circulated key compared to the true key is evidence that the circulated key is being used.
13. The non-transitory computer-readable storage medium of claim 11 , wherein identifying test takers with nearly matching test responses to the circulated key comprises:
scoring the test responses of each test taker based on the circulated key;
scoring the test responses of each test taker based on a true key; and
comparing both scores to determine if a score on the circulated key is significantly better than a score on the true key.
14. The non-transitory computer-readable storage medium of claim 11 , wherein evidence of more than one circulated key is identified and analyzed.
15. The non-transitory computer-readable storage medium of claim 11 , wherein all of the test responses for each of a plurality of test takers are from a single testing center.
16. The non-transitory computer-readable storage medium of claim 11 , wherein the plurality of test responses for each of a plurality of test takers are from more than one testing center.