Whiteboard archiving and presentation method
View Patent ↗The present invention discloses methods of archiving and optimizing lectures, presentations and other captured video for playback, particularly for blind and low vision individuals. A digital imaging device captures a preselected field of view that is subject to periodic change such as a whiteboard in a classroom. A sequence of frames is captured. Frames associated with additions or erasures to the whiteboard are identified. The Cartesian coordinates of the regions of these alterations within the frame are identified. When the presentation is played back, the regions that are altered are enlarged or masked to assist the low vision user. In another embodiment of the invention, the timing of the alterations segments the recorded audio into chapters so that the blind user can skip forward and backward to different sections of the presentation.
1. A computer-implemented process comprising the steps of setting a digital imaging device to capture a preselected field of view, the field of view having a whiteboard subject to periodic change by a foreground speaker:
capturing a sequence of frames with the digital imaging device of the whiteboard and the foreground speaker writing on the whiteboard;
capturing audio linked with the captured sequence of frames;
establishing a first timecode value associated with the initiation of writing by the foreground speaker in a first region of the whiteboard, first region identified by applying an edge detection algorithm;
establishing a second timecode value associated with the initiation of writing by the foreground speaker in a second region of the whiteboard, the second region identified by applying an edge detection algorithm;
automatically broadcasting the captured audio and displaying the first region as magnified images from the sequence of frames from the first timecode value until the second timecode value; and
automatically broadcasting the captured audio and displaying the second region as magnified images from the sequence of frames from the second timecode value.
2. A computer-implemented process comprising the steps of setting a digital imaging device to capture a preselected field of view, the field of view having a whiteboard subject to periodic change by a foreground speaker:
capturing audio linked with a captured sequence of frames;
establishing a first timecode value associated with the initiation of writing by the foreground speaker in a first region of the whiteboard, the first region identified by applying an edge detection algorithm;
establishing a second timecode value associated with the initiation of writing by the foreground speaker in a second region of the whiteboard, the second region identified by applying an edge detection algorithm;
automatically broadcasting the captured audio and displaying the first region of the whiteboard bounded by a first opaque mask overlaying the sequence of frames from the first timecode value until the second timecode value; and
automatically broadcasting the captured audio and displaying the second region of the whiteboard bounded by a second opaque mask overlaying the sequence of frames from the second timecode value.
3. The method of claim 1 further comprising the steps of:
establishing the relative position of the first region to the second region;
sequentially displaying the first and second regions according to the timecode values; and
automatically applying a visual transition associated with the relative position of the second region to the first region wherein the transition maintains an end user's spatial awareness of the whiteboard by automatically scrolling from the first magnified image to the second magnified image wherein the direction of the scrolling correlates to the relative position of the first and second regions and the timing of the scrolling correlates to the timecode values.
4. A computer-implemented process comprising the steps of:
capturing audio linked with the captured sequence of frames;
establishing a first timecode value associated with the initiation of writing by the foreground speaker in a first region of a whiteboard;
establishing a second timecode value associated with the initiation of writing by the foreground speaker in a second region of the whiteboard;
automatically broadcasting the captured audio and displaying the first region of the whiteboard with a first superimposed bounding box surrounding the first region of the whiteboard from the first timecode value until the second timecode value; and
automatically broadcasting the captured audio and displaying the second region of the whiteboard with a second superimposed bounding box surrounding the second region of the whiteboard from the second timecode value.
5. The method of claim 4 further comprising the step of applying a graphic filter within the superimposed bounding box.
6. The method of claim 1 further comprising the step of identifying the initiation of writing by the foreground speaker in the first region responsive to a predetermined geometric shape drawn by the foreground speaker and detected by the digital imaging device, the predetermined geometric shape bounding the first region.
7. A computer-implemented process comprising the steps of:
setting a digital imaging device to capture a preselected field of view, the field of view having a whiteboard subject to periodic change by a foreground speaker writing on the whiteboard;
capturing a sequence of frames with the digital imaging device according to a preselected interval;
identifying frames containing writing on the whiteboard;
identifying the regions of the writing within the identified frames based on an elapsed, predetermined time period between a first set of writings by the foreground speaker and a subsequent set of writings by the foreground speaker wherein the elapsed time permits one region to be distinguished from another region;
extracting the regions of whiteboard writings;
associating a time code sequence with the extracted regions;
applying OCR processing only to the extracted regions to produce OCR output arranged in a temporal order defined by elapsed, predetermine time periods between writings; and
assembling the OCR output according to the time code sequence of the associated extracted regions.
8. The method of claim 7 further comprising the step of exporting the assembled OCR output to an electronic document.
9. The method of claim 7 further comprising the step of exporting the assembled OCR output to speech.
10. The method of claim 7 further comprising the step of exporting the assembled OCR output to Braille.
11. The method of claim 1 further comprising the steps of:
detecting frames wherein the foreground speaker obstructs a view of writings in the first and second regions of the whiteboard; and
filtering the foreground speaker from view in the magnified images by excluding frames wherein the foreground speaker obstructs the view.
12. The method of claim 2 further comprising the steps of:
detecting frames wherein the foreground speaker obstructs a view of writings in the first and second regions of the whiteboard; and
filtering the foreground speaker from view in the masked frames by excluding frames wherein the foreground speaker obstructs the view.
13. A computer-implemented process comprising the steps of setting a digital imaging device to capture a preselected field of view, the field of view having a background subject to periodic change:
capturing a sequence of frames with the digital imaging device according to a preselected interval;
capturing audio linked with the captured sequence of frames;
establishing a first timecode value associated with the initiation of alterations in a first region of the background, the first region identified by applying an edge detection algorithm;
establishing a second timecode value associated with the initiation of alterations in a second region of the background, the second region identified by applying an edge detection algorithm;
establishing the relative position of the first region to the second region;
sequentially displaying the first and second regions according to the timecode values;
automatically broadcasting the captured audio and displaying the first region as magnified images from the sequence of frames from the first timecode value until the second timecode value;
automatically broadcasting the captured audio and displaying the second region as magnified images from the sequence of frames from the second timecode value; and
automatically applying a visual transition associated with the relative position of the second region to the first region wherein the transition maintains an end user's spatial awareness of the background by automatically scrolling from the first magnified image to the second magnified image wherein the direction of the scrolling correlates to the relative position of the first and second regions and the timing of the scrolling correlates to the timecode values.