Granted Patent B2

US 9,282,285 · App. 13/913,748 · Granted Mar 8, 2016

Abstract

Claims (69)

1. A method of providing a user video signal to an online conference, the method comprising:

receiving a live user video signal from a camera device, the live user video signal defining a field of view;

automatically identifying live initial content of a presentation region within the field of view and live initial content of a privacy region within the field of view; and

generating, as the user video signal to the online conference, a modified user video signal based on the live user video signal, the modified video signal including (i) the live initial content of the presentation region within the field of view and (ii) modified video content in place of the live initial content of the privacy region within the field of view;

wherein the camera device includes a camera and a depth sensor;

wherein receiving the live user video signal from the camera device includes acquiring, as the live user video signal, a series of video frames including captured images of a user from the camera and depth measurement data from the depth sensor; and

wherein automatically identifying the live initial content of the presentation region and the live initial content of the privacy region includes:

isolating the live initial content of the privacy region from the live initial content of the presentation region based on the depth measurement data.

2. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel having an original pixel value and a depth value, the depth values of the pixels forming at least some of the depth measurement data; and

wherein isolating the live initial content of the privacy region from the live initial content of the presentation region based on the depth measurement data includes, for each pixel, (i) maintaining the original pixel value when the depth value of that pixel is below a predefined depth threshold and (ii) replacing the original pixel value with a different pixel value when the depth value of that pixel exceeds the predefined depth threshold.

3. A method as in claim 2 wherein replacing the original pixel value with the different pixel value when the depth value of that pixel exceeds the predefined depth threshold includes:

modifying the original pixel value based on a blurring algorithm to blur the live initial content of the privacy region.

4. A method as in claim 3 wherein modifying the original pixel value based on the blurring algorithm includes:

varying a blur effect of the blurring algorithm to (i) mildly blur the live initial content of the privacy region which is closest to the presentation region, and (ii) substantially blur the live initial content of the privacy region which is farthest from the presentation region.

5. A method as in claim 4 , further comprising:

after generating the modified user video signal based on the live user video signal, receiving blur control commands which selectively disable and enable use of the blurring algorithm during subsequent operation.

6. A method as in claim 4 , further comprising:

after generating the modified user video signal based on the live user video signal, receiving blur variation commands which incrementally vary an amount of blurring performed via the blurring algorithm during subsequent operation.

7. A method as in claim 1 wherein the depth sensor is an infrared laser sensor; and wherein the depth measurement data includes infrared laser measurements of distances between the infrared laser sensor and an environment of the user.

8. A method as in claim 1 wherein the camera is a stereo camera; and wherein the depth measurement data includes depth estimate values of distances between the stereo camera and an environment of the user.

9. A method as in claim 1 wherein the camera is a webcam coupled to a user computer; and wherein the method further comprises:

rendering, by a display of the user computer, an online conference video of the online conference to enable a user of the user computer to view the online conference.

10. A method as in claim 9 wherein generating the modified video signal includes:

outputting, by the user computer, the modified video signal to an external online conference server through a computer network, the external online conference server being constructed and arranged to manage the online conference.

11. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel having an associated depth value;

wherein isolating the live initial content of the privacy region includes:

(i) dynamically establishing a depth threshold based on distance of a foreground object in the field of view from the camera device, the depth threshold being greater than the distance of the foreground object in the field of view from the camera device, and

(ii) isolating the live initial content of the privacy region from the live initial content of the presentation region based on comparison of the associated depth value of each pixel to the dynamically established depth threshold.

12. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel being constructed and arranged to have an associated depth value; and

wherein isolating the live initial content of the privacy region includes:

processing pixels lacking associated depth values as live initial content of the presentation region for inclusion in the modified video signal to prevent loss of content.

13. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel being constructed and arranged to have an associated depth value; and

wherein isolating the live initial content of the privacy region includes:

processing pixels lacking associated depth values as live initial content of the privacy region for replacement with a virtual curtain in the modified video signal to protect privacy.

14. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel being constructed and arranged to have an associated depth value; and wherein the method further comprises:

prior to generating the modified video signal, performing a preprocessing operation which associates new depth values to particular pixels initially lacking associated depth values to repair the particular pixels.

15. A method as in claim 1 wherein acquiring the series of video frames includes:

obtaining, for a particular video frame, an array of pixels that forms a particular captured image, each pixel of the array having (i) a pixel value that identifies a color for an object in the particular captured image and (ii) a depth value that defines a measured physical distance between the depth sensor and the object in the particular captured image.

16. An electronic apparatus, comprising:

a camera device;

memory; and

control circuitry coupled to the camera device and the memory, the memory storing instructions which, when carried out by the control circuitry, cause the control circuitry to:

receive a live user video signal from the camera device, the live user video signal defining a field of view,

automatically identify live initial content of a presentation region within the field of view and live initial content of a privacy region within the field of view, and

generate, as a user video signal to an online conference, a modified user video signal based on the live user video signal, the modified video signal including (i) the live initial content of the presentation region within the field of view and (ii) modified video content in place of the live initial content of the privacy region within the field of view;

wherein the camera device includes a camera and a depth sensor; and wherein the control circuitry, when receiving the live user video signal from the camera device, is constructed and arranged to:

acquire, as the live user video signal, a series of video frames including captured images of a user from the camera and depth measurement data from the depth sensor; and

wherein the control circuitry, when automatically identifying the live initial content of the presentation region and the live initial content of the privacy region, is constructed and arranged to:

isolate the live initial content of the privacy region from the live initial content of the presentation region based on the depth measurement data.

17. An electronic apparatus as in claim 16 wherein the camera is a webcam; wherein the electronic apparatus further comprises a display; and wherein the control circuitry is further constructed and arranged to:

render an online conference video of the online conference on the display to enable the user to view the online conference.

18. An electronic apparatus as in claim 16 wherein, the control circuitry, when acquiring the series of video frames, is constructed and arranged to:

obtain, for a particular video frame, an array of pixels that forms a particular captured image, each pixel of the array having (i) a pixel value that identifies a color for an object in the particular captured image and (ii) a depth value that identifies a measured physical distance between the depth sensor and the object in the particular captured image.

19. An electronic apparatus as in claim 16 wherein each video frame includes an array of pixels, each pixel having an original pixel value and a depth value, the depth values of the pixels forming at least some of the depth measurement data; and

wherein the control circuitry, when isolating the live initial content of the privacy region from the live initial content of the presentation region based on the depth measurement data is constructed and arranged to:

for each pixel, (i) maintain the original pixel value when the depth value of that pixel is below a predefined depth threshold and (ii) replace the original pixel value with a different pixel value when the depth value of that pixel exceeds the predefined depth threshold.

20. A computer program product having a non-transitory computer readable medium which stores a set of instructions to provide a user video signal to an online conference; the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of:

receiving a live user video signal from a camera device, the live user video signal defining a field of view;

automatically identifying live initial content of a presentation region within the field of view and live initial content of a privacy region within the field of view; and

generating, as the user video signal to the online conference, a modified user video signal based on the live user video signal, the modified video signal including (i) the live initial content of the presentation region within the field of view and (ii) modified video content in place of the live initial content of the privacy region within the field of view;

wherein the camera device includes a camera and a depth sensor; and

wherein receiving the live user video signal from the camera device includes acquiring, as the live user video signal, a series of video frames including captured images of a user from the camera and depth measurement data from the depth sensor; and

wherein automatically identifying the live initial content of the presentation region and the live initial content of the privacy region includes isolating the live initial content of the privacy region from the live initial content of the presentation region based on the depth measurement data.

21. A computer program product as in claim 20 wherein the camera is a webcam coupled to a user computer which contains the computerized circuitry; and wherein the method further comprises:

rendering, via a display of the user computer, an online conference video of the online conference to enable a user of the user computer to view the online conference.

22. A computer program product as in claim 20 wherein acquiring the series of video frames includes:

obtaining, for a particular video frame, an array of pixels that forms a particular captured image, each pixel of the array having (i) a pixel value that identifies a color for an object in the particular captured image and (ii) a depth value that is indicative of a measured physical distance between the depth sensor and the object in the particular captured image.

23. A computer program product as in claim 20 wherein each video frame includes an array of pixels, each pixel having an original pixel value and a depth value, the depth values of the pixels forming at least some of the depth measurement data; and

wherein isolating the live initial content of the privacy region from the live initial content of the presentation region based on the depth measurement data includes, for each pixel, (i) maintaining the original pixel value when the depth value of that pixel is below a predefined depth threshold and (ii) replacing the original pixel value with a different pixel value when the depth value of that pixel exceeds the predefined depth threshold.

Assignments (14)