System and method for evaluating a person's information technology skills
View Patent ↗An IT skills evaluation system and method for evaluating an individual's IT skills. According to one aspect of the invention, the individual is required to perform one or more practical exercises. A practical exercise is an evaluation device for evaluating not only an individual's knowledge, but also the individual's ability to apply their knowledge. One or more virtual machines are associated with each practical exercise, and the test taker must use these virtual machines to complete the exercise. In one aspect, a skills evaluation system according to one embodiment includes a testing computer on which the following software is installed: a virtual machine platform for allowing multiple virtual machines to run simultaneously on the testing computer and a practical skills evaluator for presenting exams to a test taker and scoring the exam. An exam is made up of one or more practical exercise. When the practical skills evaluator presents the practical exercise to the test taker the evaluator launches the one or more virtual machines so that the user can perform the exercise.
1. A computer implemented method for evaluating a user's information technology skills by having the user complete an information technology examination including information technology exercises using one or more virtual machines, the method comprising the steps of:
associating information technology (IT) exercises with one or more of a plurality of virtual machines;
for each of the IT exercises, pre-configuring each of the one or more of the plurality of virtual machines with at least a set of virtual machine files, wherein the set of virtual machine files determines a virtual machine state and an operating system;
for a selected IT exercise, launching the one or more of the plurality of virtual machines associated with the IT exercise in the virtual machine state and operating system, using the set of virtual machine files;
displaying to the user, information that is associated with the IT exercise, wherein the information describes tasks that the user is instructed to complete for the IT exercise, and
monitoring the tasks performed by the user on the set of virtual machine files of the one or more of the plurality of virtual machines, wherein the performed tasks place the one or more of the plurality of virtual machines into a new state; and
comparing the new state with an expected new state for the IT exercise to evaluate how well the user performed the IT exercise.
2. The method of claim 1 , further comprising:
setting a timer to expire after a certain amount of time has elapsed;
waiting for the user to complete the IT exercise and/or for the timer to expire; and
evaluating how well the user performed the IT exercise after completion or after the timer expires.
3. The method of claim 1 , wherein the one or more of the plurality of virtual machines include a first virtual machine that is configured to run a first operating system and a second virtual machine that is configured to run a second operating system different from the first operating system.
4. The method of claim 1 , wherein the tasks require the user to modify at least one particular file of the set of virtual machine files.
5. The method of claim 4 , wherein the act of evaluating how well the user performed the IT exercise comprises examining the modifications made to the at least one particular file to determine whether the particular file was modified correctly.
6. A computer implemented method for evaluating a user's information technology skills by having the user complete an information technology examination, the method comprising the steps of:
accessing a test file comprising one or more exam items;
presenting one or more exam items to the user, wherein the examination items include at least one of an information technology (IT) exercise, a multiple choice question, and an essay question, and wherein the IT exercise is associated with one or more of a plurality of virtual machines;
selecting one of the one or more exam items;
determining whether the selected exam item is an IT exercise;
launching, if the selected exam item is an IT exercise, the one or more of the plurality of virtual machines associated with the IT exercise and that are each pre-configured with at least a set of virtual machine files associated with the IT exercise, wherein the set of virtual machine files determines a virtual machine state and operating system;
displaying to the user, information that is associated with the IT exercise, wherein the information describes tasks that the user is instructed to complete for the IT exercise, and
monitoring the tasks performed by the user on the set of virtual machine files of the one or more of the plurality of virtual machines wherein the performed tasks place the one or more of the plurality of virtual machines into a new state;
comparing the new state with an expected new state for the IT exercise to evaluate how well the user performed the IT exercise; and
presenting, if the selected exam item is determined to be at least one of a multiple choice question and an essay question, the exam item to the user and waiting for a user response.
7. The method of claim 6 , further comprising:
setting a timer to expire after a certain amount of time has elapsed;
waiting for the user to complete the exercise and/or for the timer to expire; and
evaluating how well the user performed the IT exercise after completion or after the time expires.
8. The method of claim 6 , wherein the one or more of the plurality of virtual machines include a first virtual machine that is configured to run a first operating system and a second virtual machine that is configured to run a second operating system different from the first operating system.
9. The method of claim 6 , wherein the tasks require the user to modify at least one particular file of the set of virtual machine files.
10. The method of claim 9 , wherein the act of evaluating how well the user performed the IT exercise comprises examining the modifications made to the at least one particular file to determine whether the particular file was modified correctly.
11. The method of claim 9 , wherein the act of evaluating how well the user performed the IT exercise comprises examining a file that was not supposed to be modified in performing the IT exercise.
12. An information technology skills evaluation system including an information technology examination, the system comprising:
a storage medium;
a first software application stored on the storage medium for allowing multiple virtual machines to run on a single computer system;
a second software application stored on the storage medium and configured to present one or more exam items to a user, wherein the exam items include information technology (IT) exercises and at least one of a multiple choice question and an essay question, and wherein one or more of the multiple virtual machines are associated with each of the IT exercises;
a set of virtual machine files stored on the storage medium and associated with each of the IT exercises and the one or more of the multiple virtual machines, wherein each set of virtual machine files determines a virtual machine state and operating system associated with each IT exercise;
a processor for executing the first and second software applications, wherein the second software application is operable to:
launch the one or more of the multiple virtual machines associated with a selected IT exercise in the virtual machine state and operating system, using the set of virtual machine files;
display to the user, information that is associated with the IT exercise, wherein the information describes tasks that the user is instructed to complete for the IT exercise;
monitor the user performing tasks on the set of virtual machine files of the one or more of the multiple virtual machines, wherein the performed tasks place the one or more of the multiple virtual machines associated with the IT exercise into a new state; and
compare the new state with an expected new state for the IT exercise to evaluate how well the user performed the IT exercise.
13. The system of claim 12 , wherein the second software application is further operable to:
set at timer to expire after a certain amount of time has elapsed;
wait for the user to complete the IT exercise and/or for the timer to expire; and
evaluate how well the user performed the IT exercise after completion or after the timer expires.
14. The system of claim 12 , wherein the second software application comprises a test driver and a practical skills testing module (PSTM).
15. The system of claim 14 , wherein the test driver is operable to:
read a test file that comprises the one or more exam items;
select an exam item;
determine whether the selected exam item is associated with the IT exercise;
provide an exam item identifier to the PSTM if it is determined that the exam item is associated with an IT exercise, wherein the exam item identifier identifies the selected exam item; and
wait for a response from the PSTM after providing the exam item identifier to the PSTM.
16. The system of claim 14 , wherein the second software application displays the virtual machines in a testing window provided by the test driver.
17. The system of claim 12 , wherein the virtual machines include a first virtual machine that is configured to run a first operating system and a second virtual machine that is configured to run a second operating system different from the first operating system.
18. An information technology (IT) skills evaluation system for evaluating a user's IT skills including an information technology examination, the system comprising:
a test file comprising one or more exam item, wherein the one or more exam items include IT exercises and at least one of a multiple choice question and an essay question, and wherein the IT exercises are associated with one or more of a plurality of virtual machines;
a storage medium that stores a set of virtual machine files associated with each of the IT exercises and the one or more of a plurality of virtual machines, wherein each set of virtual machine files determines a virtual machine state and operating system associated with each IT exercise;
an exam item presenter that presents the one or more exam items to the user, wherein the exam item presenter comprises a virtual machine launcher that launches the one or more of the plurality of virtual machines in the virtual machine state and operating system when the IT exercise is presented to the user;
a display that displays to the user, information that is associated with the IT exercise, wherein the information describes tasks that the user is instructed to complete for the IT exercise; and
an exercise evaluator that evaluates how well the user performed the IT exercise.
19. The system of claim 18 , wherein the exercise evaluator evaluates how well the user performed the IT exercise by examining at least one of the virtual machine files.
20. The system of claim 19 , wherein in examining the at least one virtual machine file the evaluator checks a predetermined file stored within the set of virtual machine files to determine whether the user modified the file correctly.
21. The system of claim 18 , wherein the one or more of the plurality of virtual machines include a first virtual machine that is configured to run a first operating system and a second virtual machine that is configured to run a second operating system different from the first operating system.
22. A computer program product for evaluating a user's information technology skills by having the user complete an information technology examination including information technology exercises using one or more virtual machines, the computer program product being embodied in a computer readable medium and comprising computer instructions for:
associating information technology (IT) exercises with one or more of a plurality of virtual machines;
for each of the IT exercises, pre-configuring each of the one or more of the plurality of virtual machines with at least a set of virtual machine files, wherein the set of virtual machine files determines a virtual machine state and an operating system;
for a selected IT exercise, launching the one or more of the plurality of virtual machines associated with the IT exercise in the virtual machine state and operating system, using the set of virtual machine files;
displaying to the user, information that is associated with the IT exercise, wherein the information describes tasks that the user is instructed to complete for the IT exercise, and
monitoring the tasks performed by the user on the set of virtual machine files of the one or more of the plurality of virtual machines wherein the performed tasks place the one or more of the plurality of virtual machines into a new state; and
comparing the new state with an expected new state for the IT exercise to evaluate how well the user performed the IT exercise.
23. The computer program product of claim 22 , further comprising computer instructions for:
setting a timer to expire after a certain amount of time has elapsed;
waiting for the user to complete the IT exercise and/or for the timer to expire; and
evaluating how well the user performed the IT exercise after completion or after the timer expires.
24. The computer program product of claim 23 , wherein the tasks require the user to modify at least one particular file of the set of virtual machine files.
25. The computer program product of claim 24 , wherein the act of evaluating how well the user performed the IT exercise comprises computer instructions for examining the modifications made to the particular file to determine whether the at least one particular file was modified correctly.
26. The computer program product of claim 22 , wherein the one or more of the plurality of virtual machines include a first virtual machine that is configured to run a first operating system and a second virtual machine that is configured to run a second operating system different from the first operating system.
27. A computer program product for evaluating a user's information technology skills by having the user complete an information technology examination, the computer program product being embodied in a computer readable medium and comprising computer instructions for:
accessing a test file comprising one or more exam items;
presenting one or more exam items to the user, wherein the exam items include at least one of an information technology (IT) exercise, a multiple choice question, and an essay question and wherein the IT exercise is associated with one or more of a plurality of virtual machines;
selecting one of the one or more exam items;
determining whether the selected exam item is an IT exercise; and
launching, if the selected exam item is an IT exercise, the one or more of a plurality of virtual machines associated with the IT exercise and that are each pre-configured with at least a set of virtual machine files associated with the IT exercise, wherein the set of virtual machine files determines a virtual machine state and an operating system;
displaying to the user, information that is associated with the IT exercise, wherein the information describes tasks that the user is instructed to complete for the IT exercise, and
monitoring the tasks performed by the user on the set of virtual machine files of the one or more of the plurality of virtual machines wherein the performed tasks place the one or more of the plurality of virtual machines into a new state;
comparing the new state with an expected new state for the IT exercise to evaluate how well the user performed the IT exercise; and
presenting, if the selected exam item is determined to be at least one of a multiple choice question and an essay question, the exam item to the user and waiting for a user response.
28. The computer program product of claim 27 , further comprising computer instructions for:
setting a timer to expire after a certain amount of time has elapsed;
waiting for the user to complete the exercise and/or for the timer to expire; and
evaluating how well the user performed the IT exercise after completion of after the time expires.
29. The computer program product of claim 27 , wherein the tasks require the user to modify at least one particular file of the set of virtual machine files, and wherein the computer instructions for evaluating how well the user performed the IT exercise comprises computer instructions for examining the modifications made to the particular file to determine whether the particular file was modified correctly.
30. The computer program product of claim 29 , wherein the act of evaluating how well the user performed the IT exercise comprises examining a file that was not supposed to be modified in performing the IT exercise.
31. The computer program product of claim 27 , wherein the one or more of the plurality of virtual machines include a first virtual machine that is configured to run a first operating system and a second virtual machine that is configured to run a second operating system different from the first operating system.