Constructed response scoring mechanism
View Patent ↗A system, method, and related techniques are disclosed for scoring user responses to constructed response test items. The system includes a scoring engine for receiving a user response to a test question and evaluating the response against a scoring rubric. The scoring rubric may include a binding stage, an assertion stage, and a scoring stage. Furthermore, the system includes a database for referencing elements used by the scoring engine which may comprise objects, object sets, attributes of objects, and transformations of any elements.
1. A system for scoring user responses to a test question, comprising:
a scoring engine for receiving a user response to a test question and evaluating the response against a scoring rubric, wherein the scoring rubric includes a binding stage, an assertion stage, and a scoring stage; and
a database for referencing elements used by the scoring engine which include at least one of objects, object sets, attributes of objects, and transformations of any elements;
wherein the system establishes at least one reference to one or more elements during the binding stage;
the system evaluates and stores at least one assertion about an element during the assertion stage;
the system builds a scoring rubric based on said assertions;
the system assigns a score to the response based on a result produced at the assertion stage during the scoring stage; and
the bindings and the assertions use a library of relatively low-level primitives that are combined to score an array of item and item types without the need for additional programming.
2. The system of claim 1 , wherein the user response includes one or more objects comprising at least one of points, line segments connecting points, geometric objects comprised of connected line segments, and user-defined atomic objects.
3. The system of claim 2 , wherein the objects are characterized by an ordered set of points.
4. The system of claim 2 , wherein the objects have properties including at least one of locations, names, labels, and values.
5. The system of claim 1 , wherein the at least one assertion is a predicate that is either true or false.
6. The system of claim 1 , wherein named assertions are collected in a set of And-Or trees.
7. The system of claim 6 , wherein there is one tree for each numeric score point.
8. The system of claim 1 , wherein the user response includes one or more objects comprising natural language text, wherein the natural language text is parsed to yield an object subset in the binding stage.
9. The system of claim 1 , wherein the user response includes one or more objects.