Employee selection via adaptive assessment
An employee can be selected (e.g., employee job performance can be predicted) via a predictive model. Items presented as part of an assessment can be chosen according to which has greatest predictive power. The next item to be presented can be selected based on imputation of inputs to the predictive model for items not yet presented. Expected reduction in estimated output variance can be calculated.
1 . A method comprising:
administering an assessment to a candidate employee;
receiving an answer to at least one question presented to the candidate employee during administration of the assessment;
based on the answer to the at least one question, selecting, during administration of the assessment, in view of the answer to the at least one question, a next question out of a set of possible questions for presentation to the candidate employee based on an expectation of reduction in assessment output variance if the next question were to be answered;
presenting the next question to the candidate employee; and
outputting at least one assessment output.
2 . The method of claim 1 wherein the expectation of reduction in assessment output variance is determined by applying plausible values to at least one of a plurality of inputs to a predictive model for one or more respective questions not yet answered by the candidate employee while constraining an other of the inputs for a question not yet answered by the candidate employee.
3 . The method of claim 2 wherein:
the plausible answers are chosen at random according to an observed distribution of answers for one or more questions by other candidate employees.
4 . The method of claim 3 wherein different sets of random answers for questions not yet answered are applied to a neural network to estimate output variance.
5 . The method of claim 2 wherein the expectation of reduction in assessment output variance is calculated as a weighted average for a plurality of possible answers to the constrained input.
6 . The method of claim 2 wherein the predictive model comprises a neural network.
7 . The method of claim 6 wherein:
fewer than all inputs are available to the neural network; and
an output value for the neural network is used to calculate one or more of the at least one assessment outputs.
8 . The method of claim 2 wherein expectation of reduction in assessment output variance if the next question were to be answered is calculated for a group of questions designated as for determining a latent trait.
9 . The method of claim 1 wherein a value for the latent trait is used as an input to a predictive model for calculating one or more of the at least one assessment outputs.
10 . The method of claim 1 further comprising:
electronically receiving answers to one or more biographical questions to the candidate employee;
wherein the next question is selected based at least on the answers to the one or more biographical questions.
11 . The method of claim 1 further comprising:
stopping the assessment when the expectation of reduction in assessment output variance drops below a threshold.
12 . One or more computer-readable media comprising computer-executable instructions for performing the method of claim 1 .
13 . A method comprising:
for a set of a plurality inputs to a predictive model operable to output an assessment output, applying random values to one or more of the inputs and observing a resulting first variance in the output;
constraining at least one of the one or more inputs while applying random values to other of the one or more of the inputs and observing a resulting second variance in the output;
calculating a reduction in variance; and
based on the reduction of variance, selecting a question associated with the input for presentation to a job applicant during an assessment.
14 . The method of claim 13 wherein:
the constraining comprises constraining the at least one of the one or more inputs to respective possible answers for the at least one input of the one or more inputs;
the calculating a reduction in variance comprises estimating variances for the respective possible answers; and
the calculating a reduction in variance further comprises estimating the second variance in the output via a weighted average of the variances for the respective possible answers.
15 . A method comprising:
administering an assessment to a candidate employee, wherein the assessment outputs at least one assessment output; and
during the assessment, choosing a next question to present to the candidate employee based on answers to one or more other questions already presented during the assessment;
wherein the assessment output is based on a value indicative of a measure of at least one personality trait for the candidate employee relative to other candidate employees already tested.
16 . The method of claim 15 wherein choosing the next question comprises determining which question would reduce estimated variance most if the answer to the question were available.
17 . A method comprising:
identifying an item out of a set of possible items as having greater predictive power than an other item out of the set of possible items; and
presenting the item as part of a job effectiveness assessment for response by a candidate employee.
18 . The method of claim 17 , wherein:
the identifying comprises measuring sensitivity of a predictive model for an item not yet presented.
19 . The method of claim 18 , wherein:
the identifying further comprises choosing an item for which the predictive model exhibits a greater sensitivity.
20 . The method of claim 17 , wherein:
the identifying comprises applying possible responses to a predictive model for an item not yet presented.
21 . The method of claim 20 , wherein:
the identifying further comprises measuring change in prediction by the model across the possible responses for the item not yet presented.
22 . The method of claim 21 , wherein:
the identifying further comprises choosing an item having a greater change in prediction.
23 . An adaptive assessment tool comprising:
means for collecting answers to questions from a candidate employee;
means for choosing a question from a set of possible questions according to an adaptive selection technique based on previous answers to questions by the candidate employee, whereby the question is a chosen question;
means for administering one or more administered questions, wherein the means for administering is responsive to the means for choosing and is configured to administer the chosen question; and
means for indicating an assessment result of the candidate employee based on answers by the candidate employee to the one or more administered questions.