Generating probing questions to test attention to automated educational materials
A method, apparatus and program product utilize an automated educational tool to generate or modify one or more multiple choice questions in a question set in connection with tracking a user's attention state to dynamically vary the difficulty of a user assessment performed by the automated educational tool and thereby encourage the user to pay greater attention to the material being presented to the user.
1. A method of presenting educational material, the method comprising:
using at least one processor, analyzing educational material to be presented to a user and generating therefrom one or more inattentive user assessment questions associated with the educational material;
using at least one processor, presenting the educational material to the user;
using at least one processor, performing a user assessment during presentation of the educational material to assess the user's understanding of the educational material;
using at least one processor, monitoring user attention during presentation of the educational material;
in response to detection of an inattentive state for the user when monitoring user attention, dynamically increasing a difficulty of the user assessment by presenting the one or more inattentive user assessment questions to the user; and
evaluating the one or more inattentive user assessment questions by presenting the one or more inattentive user assessment questions to a second user not determined to be in an inattentive state.
2. The method of claim 1 , wherein generating the one or more inattentive user assessment questions is performed dynamically during presentation of the educational material to the user.
3. The method of claim 1 , wherein generating the one or more inattentive user assessment questions is performed prior to presentation of the educational material to the user.
4. The method of claim 1 , wherein generating the one or more inattentive user assessment questions includes generating a new question from the educational material.
5. The method of claim 4 , wherein generating the new question includes:
parsing the educational material;
extracting one or more entities and one or more relationships from the parsed educational material; and
generating the new question based upon the extracted one or more entities and one or more relationships.
6. The method of claim 4 , wherein generating the new question includes applying at least a portion of the educational material to a question answer pipeline.
7. The method of claim 1 , wherein generating the one or more inattentive user assessment questions includes modifying an existing question included in the educational material.
8. The method of claim 7 , wherein modifying the existing question includes making the existing question more difficult.
9. The method of claim 7 , wherein modifying the existing question includes generating one or more near-match distractor answers.
10. The method of claim 7 , wherein modifying the existing question includes generating one or more wrong answers similar to one or more correct answers.
11. The method of claim 7 , wherein modifying the existing question includes extracting a number and polarity of correct answers to the existing question.
12. The method of claim 7 , wherein modifying the existing question includes training a machine learning model to predict answer correctness in a question set associated with the educational material.
13. The method of claim 1 , wherein monitoring user attention includes tracking user interaction with a mouse, tracking user interaction with a keyboard, tracking user interaction with a touchscreen display, tracking user eye movement, tracking user interaction with one or more applications, or tracking user interaction with one or more windows.
14. The method of claim 1 , wherein dynamically increasing the difficulty of the user assessment includes selecting the one or more inattentive user assessment questions from a question set.
15. The method of claim 1 , wherein dynamically increasing the difficulty of the user assessment includes dynamically generating the one or more inattentive user assessment questions as modifications of one or more existing questions in a question set.
16. The method of claim 1 , further comprising:
presenting the one or more inattentive user assessment questions to the user during the user assessment;
receiving one or more answers from the user; and
taking a corrective action in response to the one or more answers received from the user being incorrect.
17. The method of claim 16 , wherein taking the corrective action includes restarting a predetermined portion of the presentation.
18. An apparatus, comprising:
at least one processor; and
program code configured upon execution by the at least one processor to present educational material by:
analyzing educational material to be presented to a user and generating therefrom one or more inattentive user assessment questions associated with the educational material;
presenting the educational material to the user;
in response to detection of an inattentive state for the user when monitoring user attention, dynamically increasing a difficulty of the user assessment by presenting the one or more inattentive user assessment questions to the user; and
evaluating the one or more inattentive user assessment questions by presenting the one or more inattentive user assessment questions to a second user not determined to be in an inattentive state.
19. A program product, comprising:
a non-transitory computer readable medium; and
program code stored on the non-transitory computer readable medium and configured upon execution by at least one processor to present educational material by:
analyzing educational material to be presented to a user and generating therefrom one or more inattentive user assessment questions associated with the educational material;
presenting the educational material to the user;
performing a user assessment during presentation of the educational material to assess the user's understanding of the educational material;
monitoring user attention during presentation of the educational material;
in response to detection of an inattentive state for the user when monitoring user attention, dynamically increasing a difficulty of the user assessment by presenting the one or more inattentive user assessment questions to the user; and
evaluating the one or more inattentive user assessment questions by presenting the one or more inattentive user assessment questions to a second user not determined to be in an inattentive state.