Granted Patent B2

US 9,762,855 · App. 14/644,494 · Granted Sep 12, 2017

Abstract

Claims (46)

1. A method of sharing physical target boards in electronic conferences, the method comprising:

capturing video by a camera of a computing machine participating in an electronic conference, the camera oriented to include a physical target board in its field of view;

transforming the captured video to generate output video, the output video providing an enhanced view of the physical target board; and

transmitting the output video to a set of other computing machines participating in the electronic conference, to enable a user of each of the set of other computing machines to see the enhanced view of the physical target board from the respective computing machine,

wherein the physical target board is oriented at an angle relative to the camera such that the camera does not have a direct, face-on view of the physical target board, and wherein transforming the captured video includes performing a perspective deskew operation to render images of the physical target board in the output video from a face-on perspective, and

wherein transforming the captured video further includes:

identifying a set of edges of the physical target board; and

overlaying a border that follows the set of edges to visually accentuate the physical target board in the output video.

2. The method of claim 1 , wherein transforming the captured video further includes at least one of (i) rendering video content of neighboring objects adjacent to the physical target board with reduced contrast and/or color saturation and (ii) rendering the physical target board with increased brightness and/or color saturation.

3. The method of claim 1 , wherein transforming the captured video further includes removing video content showing neighboring objects of the physical target board, such that the physical target board appears alone in the output video.

4. The method of claim 3 , wherein transforming the captured video further includes:

identifying a background of the physical target board on which markings are made, the background having variations in color and brightness; and

rendering the background, including the variations in color and brightness, with uniform color and brightness.

5. The method of claim 1 , further comprising, prior to transforming the captured video, performing a board detection operation on the captured video to identify the physical target board within the field of view of the camera.

6. The method of claim 5 , wherein performing the board detection operation includes:

identifying a set of quadrangles within the field of view of the camera; and

selecting a quadrangle, from the identified set of quadrangles, that has characteristics that most closely match a set of predetermined expected characteristics of physical target boards.

7. The method of claim 6 , further comprising accepting user input to assist manually in identifying the physical target board within the field of view of the camera.

8. The method of claim 1 , wherein transforming the captured video further includes:

generating, from the captured video, a board model that provides a representation of a set of markings made on the physical target board,

wherein the method further comprises superimposing the board model with live content from the captured video to generate combined content, the combined content showing the set of markings overlaid with any occluding object that blocks the set of markings.

9. The method of claim 8 , wherein the act of superimposing includes computing a weighted sum of the board model with the live video content, wherein the board model has a greater weight in the weighted sum than does the live video content.

10. The method of claim 9 , wherein transforming the captured video further includes, in response to an occluding object moving to reveal changes in the set of markings that were previously hidden behind the occluding object, updating the board model to reflect the revealed changes in the set of markings,

wherein the method further includes superimposing the updated board model with the live video content.

11. The method of claim 9 , further comprising:

computing a delta image based on a difference between the live video content and the board model;

generating a large blob mask from the delta image, the large blob mask having a first value at a set of locations where occluding image content is present in the delta image and a second value at a set of locations where no occluding image content is present in the delta image; and

updating the board model based on the large blob mask, including adding the live video content to content of the board model at the set of locations where the large blob mask has the second value.

12. The method of claim 11 , wherein the delta image includes a feature corresponding to an occluding object, the feature having an initial outline, and wherein generating the large blob mask includes performing a blurring operation and a thresholding operation based on the delta image, to cause the initial outline to become smoother.

13. The method of claim 11 , wherein the delta image includes a feature corresponding to an occluding object, and wherein generating the large blob mask includes:

performing a convex hull operation on the feature, the convex hull operation circumscribing the feature within a fully-convex polygon; and

changing the value at one or more locations within the fully-convex polygon from the second value to the first value.

14. The method of claim 11 , wherein the delta image includes a feature corresponding to an occluding object, and wherein generating the large blob mask includes:

analyzing the delta image to determine whether the feature intersects a location in the delta image that corresponds to an edge of the physical target board; and

in response to determining that the feature does not intersect any location corresponding to an edge of the physical target board, adding the feature to the board model.

15. The method of claim 11 , wherein transforming the captured video further includes:

monitoring pixels of the live video over time; and

updating the board model to include any pixels of the live video that remain substantially constant for a predetermined period of time.

16. The method of claim 11 , further comprising generating a summary series of still images of the output video that capture a sequence of changes in markings on the physical target board over a course of the electronic conference, wherein generating the summary series includes:

taking a first still image of the output video in response to detecting that multiple new marks were sequentially made on the physical target board and then that an erasure was made; and

taking a second still image of the output video in response to detecting that multiple new erasures were sequentially made from the physical target board and then that a new mark was made; and

providing the first still image and the second still image in the summary series.

17. The method of claim 8 , wherein transmitting the output video to the set of other computing machines includes providing the output video in two video streams:

a first video stream conveying the board model; and

a second video stream conveying live content from the captured video,

wherein superimposing the board model with live content from the captured video is performed by each of the set of other computing machines by combining the first video stream with the second video stream.

Assignments (14)