Segmenting and playback of whiteboard video capture
View Patent ↗The present invention discloses methods of archiving and optimizing lectures, presentations and other captured video for playback, particularly for blind and low vision individuals. A digital imaging device captures a preselected field of view that is subject to periodic change such as a whiteboard in a classroom. A sequence of frames is captured. Frames associated with additions or erasures to the whiteboard are identified. The Cartesian coordinates of the regions of these alterations within the frame are identified. When the presentation is played back, the regions that are altered are enlarged or masked to assist the low vision user. In another embodiment of the invention, the timing of the alterations segments the recorded audio into chapters so that the blind user can skip forward and backward to different sections of the presentation.
1. A computer-implemented method comprising the steps of setting a digital imaging device to capture a preselected field of view, the field of view having a whiteboard subject to periodic change:
automatically initiating a step of capturing a sequence of frames with the digital imaging device of the whiteboard;
automatically initiating a step of capturing audio linked with the captured sequence of frames;
establishing a first timecode value associated with the initiation of writing in a first region of the whiteboard;
establishing a second timecode value associated with the initiation of writing in a second region of the whiteboard;
automatically establishing the relative position of the first region to the second region;
automatically broadcasting the captured audio and displaying the first region from the sequence of frames from the first timecode value until the second timecode value; and
automatically broadcasting the captured audio and displaying the second region from the sequence of frames from the second timecode value wherein a scroll transition associated with the relative position of the first region to the second region is applied to show the spatial relationship between first and second regions.
2. The method of claim 1 further comprising the step of providing end user control to manually scroll to different sections of the canvas.
3. The method of claim 1 wherein the first and second regions automatically broadcast are magnified.
4. The method of claim 1 wherein the first and second regions automatically broadcast are bounded by opaque masks.
5. The method of claim 1 wherein the first and second regions automatically broadcast are surrounded by bounding boxes.
6. The method of claim 1 further comprising the step of identifying the initiation of writing in a second region of the whiteboard responsive to a predetermined geographic shape detected by the digital imaging device, the predetermined geometric shape marked proximate to the first region.
7. The method of claim 1 further comprising the step of identifying the first and second regions of the whiteboard by applying an edge detection algorithm.
8. A computer-implemented method comprising the steps of setting a digital imaging device to capture a preselected field of view, the field of view having a whiteboard subject to periodic change by a foreground speaker:
automatically initiating a step of capturing a sequence of frames with the digital imaging device of the whiteboard and the foreground speaker writing on the whiteboard;
automatically initiating a step of capturing audio linked with the captured sequence of frames;
establishing a first timecode value associated with the initiation of writing by the foreground speaker in a first region of the whiteboard;
establishing a second timecode value associated with the initiation of writing by the foreground speaker in a second region of the whiteboard, the second region indicative of a new discussion topic;
timecoding the captured frames and audio;
automatically segmenting the captured audio into chapters defined by timecode values associated with the initiation of writing in the second region whereby the process assumes the second region is indicative of a new discussion topic;
automatically displaying only the first region from the sequence of frames and broadcasting a first audio chapter associated therewith; and
automatically displaying only the second region from the sequence of frames and broadcasting a second audio chapter associated therewith.
9. The method of claim 8 further comprising the step of identifying the initiation of writing in a second region of the whiteboard responsive to a predetermined geographic shape detected by the digital imaging device, the predetermined geometric shape marked proximate to the first region.
10. The method of claim 8 further comprising the step of identifying the first and second regions of the whiteboard by applying an edge detection algorithm.
11. A computer-implemented method comprising the steps of setting a digital imaging device to capture a preselected field of view, the field of view having a whiteboard subject to periodic change:
automatically initiating a step of capturing a sequence of frames with the digital imaging device of the whiteboard;
automatically initiating a step of capturing audio linked with the captured sequence of frames;
establishing a first timecode value associated with the initiation of writing in a first region of the whiteboard;
establishing a second timecode value associated with the initiation of writing in a second region of the whiteboard, the second region indicative of a new discussion topic;
automatically segmenting the captured audio and frame sequence into chapters defined by timecode values associated with the initiation of writing in the second region whereby the process assumes the second region is indicative of a new discussion topic;
generating a chapter menu comprising a plurality of thumbnail images;
responsive to an end user selection of a first thumbnail image, automatically displaying only the first region from the sequence of frames and broadcasting a first audio chapter associated therewith; and
responsive to the end user selection of a second thumbnail image, automatically displaying only the second region from the sequence of frames and broadcasting a second audio chapter associated therewith.
12. The method of claim 11 wherein a thumbnail image representative of the first region are generated from one or more frame sequences of the first region set by a timecode value substantially concurrent or after the initiation of writing in the second region.
13. The method of claim 12 wherein end-user selection of the thumbnail image initiates playback of the associated segment of audio and corresponding frames starting with the initiation of writing in the first region.
14. The method of claim 11 wherein frames of the segment played back are magnified.
15. The method of claim 11 wherein frames of the segment played back are bounded by opaque masks.
16. The method of claim 11 wherein frames of the segment played back are surrounded by bounding boxes.
17. The method of claim 11 further comprising the step of identifying the initiation of writing in a second region of the whiteboard responsive to a predetermined geographic shape detected by the digital imaging device, the predetermined geometric shape marked proximate to the first region.
18. The method of claim 11 further comprising the step of identifying the first and second regions of the whiteboard by applying an edge detection algorithm.