Data extraction from temporal image data
View Patent ↗In a method and system to extract data from handwritten information when the information is captured as sequences of strokes, vectors, or marks, temporal data is stored within the color or gray-scale encoding of pixel values within a bitmapped image. A bitmapped image is used to create a derived image suitable for data extraction and a method of analyzing the image, including the encoded temporal data, to yield an adjusted image that better reflects the intent of the person who created the handwriting. While the primary application of the invention is in processing handwritten responses to assessment items, the system and method can be advantageously used to extract data from or to improve the presentation of information from an electronic representation of temporal human marks in many applications. Additionally, the system and method can be advantageously used to store and process other types of data within a bitmapped image.
1. A computerized method of embedding temporal data for handwritten information within a bitmapped image of the handwritten information, said method comprising the steps of:
creating an electronic representation for each stroke, vector, or mark of the handwritten information;
creating a bitmapped image of the handwritten information, the bitmapped image having background and non-background pixels, wherein at least one non-background pixel within the bitmapped image has a color or gray scale value and corresponds to a point within the electronic representation; and
using a computer to store a temporal data value within the color or gray scale value for the non-background pixel.
2. The computerized method of claim 1 in which at least some of the handwritten information represents at least one response to a stimulus in an assessment.
3. The computerized method of claim 2 in which the at least one response is a constructed response.
4. The computerized method of claim 2 in which the at least one response is one or more strokes, vectors, or marks that indicate a selection to a response item.
5. The computerized method of claim 1 in which the temporal data is a function of a sequential position of the stroke, vector or mark in the handwritten information.
6. The computerized method of claim 5 in which the temporal data is scaled.
7. The computerized method of claim 1 in which the temporal data is a function of a time at which the electronic representation was created.
8. The computerized method of claim 7 , in which the temporal data is scaled.
9. The computerized method of claim 1 in which the background pixel color is white.
10. The computerized method of claim 1 in which erasure vectors are represented by color or gray scale values set to white.
11. The computerized method of claim 1 , wherein the color or gray scale value is defined by a plurality of bits in a digital representation of the non-background pixel and the step of using a computer to store a temporal data value within the color or gray scale value for the non-background pixel comprises the steps of:
assigning the color or gray scale value to at least one bit in the digital representation of the non-background pixel; and
assigning the temporal data to remaining bits in the digital representation of the non-background pixel.
12. The computerized method of claim 11 , wherein the color or gray scale value is assigned to high order bits in the digital representation of the non-background pixel and the temporal data is assigned to low order bits in the digital representation of the non-background pixel.
13. The computerized method of claim 1 , wherein the color or gray scale value is defined by a plurality of bits in a digital representation of the non-background pixel and wherein additional information other than temporal data is stored in the at least one non-background pixel, said method further comprising the steps of:
assigning at least one numerical value to the additional information;
assigning the at least one numerical value to at least one bit in the digital representation of the non-background pixel; and
assigning the temporal data to remaining bits in the digital representation of the non-background pixel.
14. The computerized method of claim 13 , wherein the additional information is an intensity of the stroke, vector or mark, a type of instrument that made a stroke, vector or mark, physical pressure exerted on an instrument that made a stroke, vector or mark, or a number of strokes, vectors or marks that were made for the non-background pixel.
15. A computerized method of creating a derived image, comprising the steps of:
embedding temporal data for handwritten information within a bitmapped image of handwritten information according to the method of claim 1 ; and
creating a derived image of all non-background pixels represented without temporal data.
16. The computerized method of claim 15 , further comprising the step of extracting data from the derived image.
17. The computerized method of claim 16 , wherein the step of extracting data is conducted manually or automatically and includes at least one of key-from-image, human evaluation or scoring, character recognition, conversion to text, and scoring of selected responses.
18. A computerized method of representing handwritten information in a manner that represents the intent of a person who created the handwritten information, comprising the steps of:
embedding temporal data for the handwritten information according to the method of claim 1 ;
processing the electronic representations in the same sequence as a temporal sequence of the creation of the stroke, vector, or mark so that each non-background pixel within the bitmapped image is represented by a last temporal stroke, vector, or mark of handwritten information which includes the point corresponding to the non-background pixel; and
creating an adjusted bitmapped image by selectively including or excluding the electronic representations based on their temporal sequence.
19. The computerized method of claim 18 , wherein the step of creating the adjusted bitmapped image further comprises the step of creating a display image of the bitmapped image in which an analyst can see false color representations that represent the temporal sequence of the electronic representations for either the entire display image or for selected regions of the display image.
20. The computerized method of claim 19 in which the analyst can further identify specific instances or ranges of temporal data values for inclusion or exclusion in the display image.
21. The computerized method of claim 18 , wherein the method is performed automatically by a computer system.
22. The computerized method of claim 1 , wherein the stroke, vector or mark is digitally stored in a writing instrument before the electronic representation is created.
23. The computerized method of claim 22 , wherein the writing instrument is a digital pencil, a digital pen, a digital stylus, a digital tablet, a digital notepad, a digitizing white board, a personal digital assistant having a stylus or a touch-sensitive screen, or a computer system with a mouse, a stylus, or a touch-sensitive screen.
24. A computer readable program storage medium embodying instructions executable by a computer system for embedding temporal data for handwritten information within a bitmapped image of the handwritten information, the instructions being executable by the computer system to:
read an electronic representation for each stroke, vector, or mark of the handwritten information into main memory or secondary memory of the computer system;
create a bitmapped image of the handwritten information, the bitmapped image having background and non-background pixels, wherein at least one non-background pixel within the bitmapped image has a color or gray scale value and corresponds to a point within the electronic representation; and
store a temporal data value within the color or gray scale value for the non-background pixel.
25. The computer readable program storage medium according to claim 24 , wherein the main memory is random access memory.
26. The computer readable program storage medium according to claim 24 , wherein the secondary memory is a hard disk drive or a removable storage device.
27. The computer readable program storage medium according to claim 26 , wherein the removable storage device is an optical disk, a read-write CD-ROM, a read-write DVD-ROM, a floppy disk, a magnetic tape, a cartridge, a memory chip, a “Memory Stick,” a flash card, a microdrive, or a USB memory module.
28. The computer readable program storage medium according to claim 24 , wherein the stroke, vector, or mark is digitally stored in a writing instrument.
29. The computer readable program storage medium according to claim 28 , wherein the writing instrument is a digital pencil, a digital pen, a digital stylus, a digital tablet, a digital notepad, a digitizing white board, a personal digital assistant having a stylus or a touch-sensitive screen, or a computer system with a mouse, a stylus, or a touch-sensitive screen.
30. The computer readable program storage medium according to claim 28 , wherein the writing instrument is not connected to the computer system when the stroke, vector or mark is digitally stored in the writing instrument.
31. The computer readable program storage medium according to claim 28 , wherein the writing instrument is connected to the computer system when the stroke, vector or mark is digitally stored in the writing instrument.
32. The computer readable program storage medium according to claim 28 , wherein the writing instrument creates the electronic representation.
33. The computer readable program storage medium according to claim 28 , wherein the computer system creates the electronic representation.
34. The computer readable program storage medium according to claim 24 , wherein at least some of the handwritten information represents at least one response to a stimulus in an assessment.
35. The computer readable program storage medium according to claim 34 , wherein the at least one response is a constructed response.
36. The computer readable program storage medium according to claim 34 , wherein the at least one response is one or more strokes, vectors, or marks that indicate a selection to a response item.
37. The computer readable program storage medium according to claim 24 , wherein the temporal data is a function of a sequential position of the stroke, vector or mark in the handwritten information.
38. The computer readable program storage medium according to claim 37 , wherein the temporal data is scaled.
39. The computer readable program storage medium according to claim 24 , wherein the temporal data is a function of a time at which the electronic representation was created.
40. The computer readable program storage medium according to claim 39 , wherein the temporal data is scaled.
41. The computer readable program storage medium according to claim 24 , wherein the background pixel color is white.
42. The computer readable program storage medium according to claim 24 , wherein erasure vectors are represented by color or gray scale values set to white.
43. The computer readable program storage medium according to claim 24 , wherein the instructions are further executable by the computer system to:
define the color or gray scale value by a plurality of bits in a digital representation of the non-background pixel; and
store a temporal data value within the color or gray scale value for the non-background pixel by assigning the color or gray scale value to at least one bit in the digital representation of the non-background pixel and assigning the temporal data to remaining bits in the digital representation of the non-background pixel.
44. The computer readable program storage medium according to claim 43 , wherein the color or gray scale value is assigned to high order bits in the digital representation of the non-background pixel and the temporal data is assigned to low order bits in the digital representation of the non-background pixel.
45. The computer readable program storage medium according to claim 24 , wherein the color or gray scale value is defined by a plurality of bits in a digital representation of the non-background pixel and wherein additional information other than temporal data is stored in the at least one non-background pixel, the instructions further executable by the computer system to:
assign at least one numerical value to the additional information;
assign the at least one numerical value to at least one bit in the digital representation of the non-background pixel; and
assign the temporal data to remaining bits in the digital representation of the non-background pixel.
46. The computer readable program storage medium according to claim 45 , wherein the at least one numerical value is assigned to high order bits in the digital representation of the non-background pixel and the temporal data is assigned to low order bits in the digital representation of the non-background pixel.
47. The computer readable program storage medium according to claim 45 , wherein the additional information is an intensity of the stroke, vector or mark, a type of instrument that made a stroke, vector or mark, physical pressure exerted on an instrument that made a stroke, vector or mark, or a number of strokes, vectors or marks that were made for the non-background pixel.
48. A computer readable program storage medium embodying instructions executable by a computer system for creating a derived composite image, the instructions being executable by the computer system to:
embed temporal data for handwritten information within a bitmapped image of handwritten information according to claim 24 ; and
create a derived composite image of all non-background pixels represented without temporal data.
49. The computer readable program storage medium according to claim 48 , wherein the instructions are further executable by the computer system to extract data from the derived composite image.
50. The computer readable program storage medium according to claim 49 , wherein the data includes at least one of key-from-image, human evaluation or scoring, character recognition, conversion to text, and scoring of selected responses.
51. A computer readable program storage medium embodying instructions executable by a computer system for representing handwritten information in a manner that represents the intent of a person who created the handwritten information, the instructions being executable by the computer system to:
embed temporal data for the handwritten information according to claim 24 ;
process the electronic representations in the same sequence as a temporal sequence of the creation of the stroke, vector, or mark so that each non-background pixel within the bitmapped image is represented by a last temporal stroke, vector, or mark of handwritten information which includes the point corresponding to the non-background pixel; and
create an adjusted bitmapped image by selectively including or excluding the electronic representations based on their temporal sequence.
52. The computer readable program storage medium according to claim 51 , wherein the instructions are further executable by the computer system to create a display image of the bitmapped image in which an analyst can see false color representations that represent the temporal sequence of the electronic representations for either the entire display image or for selected regions of the display image.
53. The computer readable program storage medium according to claim 52 , wherein the analyst can further identify specific instances or ranges of temporal data values for inclusion or exclusion in the display image.