IP Library Granted Patent US 11,695,819
Granted Patent B2
US 11,695,819 · App. 17/456,975 · Granted Jul 4, 2023

Securing image data from unintended disclosure at a videoconferencing endpoint

Inventors: Kui Zhang (Round Rock, TX); Varun Ajay Kulkarni (Cedar Park, TX); Raghavendra Balavalikar Krishnamurthy (Austin, TX); Rajen Bhatt (McDonald, PA); Stephen Schaefer (Austin, TX)
Assignee: PLANTRONICS, INC.
H04L65/762G06V20/46H04L65/403H04N1/4493
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,695,819
App. No.
17/456,975
Granted
Jul 4, 2023
Kind
B2
Abstract

A system for preventing private image data captured at an endpoint from being shared during a videoconference is provided. A user can select three-dimensional regions which will not be seen during a videoconference while areas in front of the designated regions remain viewable.

Claims (91)

1. A computer-implementable method for securing image data from unintended disclosure at a videoconferencing endpoint configured to detect meeting participants, comprising:

capturing a first image data frame using an image capture device of a videoconferencing endpoint, the image capture device having a field of view, the first image data frame depicting a scene at the videoconferencing endpoint;

extracting, using a processor at the videoconferencing endpoint, visual patterns of the scene from the first image data frame;

generating, using the processor, a three-dimensional model corresponding to the scene;

determining one or more planes of the three-dimensional model using the processor at the videoconferencing endpoint;

designating a plane from among the one or more planes as a boundary plane based, at least in part, on the visual patterns of the first image data frame;

displaying a depiction of the three-dimensional model using an electronic display device, the depiction of the three-dimensional model depicting the boundary plane as a selectable polygon;

designating, based on a selection of the selectable polygon, a region which is behind the boundary plane as a region of exclusion, whereby image data emanating from the region of exclusion is excluded from consideration as being indicative of a meeting participant.

2. The method of claim 1 , further comprising excluding captured image data emanating from within the region of exclusion from transmission to a remote endpoint.

3. The method of claim 1 , wherein designating the plane as a boundary plane using the processor comprises executing one or more monocular depth estimation algorithms.

4. The method of claim 1 , wherein designating the plane as a boundary plane further using the processor comprises executing one or more line segmentation algorithms.

5. The method of claim 1 , further comprising:

capturing a second image data frame using the image capture device of the videoconferencing endpoint, wherein capturing the second image data frame comprises capturing image data emanating from within the region of exclusion;

modifying the second image data frame, wherein modifying the second image data frame comprises replacing the image data emanating from within the region of exclusion with other visual data; and

transmitting the second image data frame to a remote endpoint as part of a visual data stream.

6. The method of claim 1 , further comprising:

capturing a second image data frame using the image capture device of the videoconferencing endpoint, wherein capturing the second image data frame comprises capturing image data corresponding to the polygon;

modifying the second image data frame, wherein modifying the second image data frame comprises replacing the image data corresponding to the polygon with other visual data; and

transmitting the second image data frame to a remote endpoint as part of a visual data stream.

7. The method of claim 6 , wherein the image data emanating from within the region of exclusion corresponds to a reflection of a person located within the field of view and located outside the region of exclusion.

8. The method of claim 1 , further comprising:

detecting image data indicative of a first person within the field of view;

determining that the image data indicative of the first person corresponds to a location outside the region of exclusion; and

transmitting the image data indicative of the first person to a remote endpoint as part of a visual data stream.

9. The method of claim 1 , further comprising:

detecting image data indicative of a head within the polygon;

determining that the image data indicative of the head corresponds to a pixel height below a threshold corresponding to the boundary plane; and

excluding the image data indicative of the head from transmission to a remote endpoint as part of a visual data stream.

10. The method of claim 1 , further comprising:

detecting image data indicative of a head within the polygon;

determining that the image data indicative of the head corresponds to a pixel height exceeding a threshold corresponding to the boundary plane; and

determining, responsive to determining that the image data indicative of the head corresponds to a pixel height exceeding the threshold corresponding to the boundary plane, that the image data indicative of the head corresponds to image data indicative of a face.

11. The method of claim 1 , further comprising:

detecting a meeting participant at a first location at the videoconferencing endpoint;

framing the meeting participant at the first location within a first frame;

including data of the first frame within a visual stream for transmission to a remote endpoint;

tracking the meeting participant to a second location at the videoconferencing endpoint, the second location closer to the image capture device than the boundary plane;

framing the meeting participant at the second location within a second frame;

including data of the second frame corresponding to the meeting participant within the visual stream; and

excluding data of the second frame not corresponding to the meeting participant from the visual stream.

12. The method of claim 11 , wherein excluding data of the second frame not corresponding to the meeting participant comprises replacing the data of the second frame not corresponding to the meeting participant with other visual data.

13. A videoconferencing system, comprising:

a processor;

a display device;

an image capture device configured to capture a series of visual data frames for processing by the processor;

a memory configured to store one or more visual data frames;

a network interface configured to transmit visual data frames to a remote endpoint;

a data bus coupled to the processor, the display device, the image capture device, the network interface, and the memory; and

a non-transitory, computer-readable storage medium, coupled to the data bus, and storing instructions executable by the processor, the instructions comprising instructions to:

capture a first image data frame using the image capture device, the first image data frame depicting a scene;

extract visual patterns of the scene from the first image data frame;

generate a three-dimensional model corresponding to the scene;

determine one or more planes of the three-dimensional model;

designate a plane from among the one or more planes as a boundary plane based, at least in part, on the visual patterns of the first image data frame;

display a depiction of the three-dimensional model using an electronic display device, the depiction of the three-dimensional model depicting the boundary plane as a selectable polygon;

receive, at the processor, a selection corresponding to the polygon; and

designate a region which is behind the plane as a region of exclusion, whereby a person located in the region of exclusion is excluded from consideration as a possible meeting participant.

14. The videoconferencing system of claim 13 , the instructions further comprising instructions to exclude captured image data emanating from within the region of exclusion from transmission to a remote endpoint.

15. The videoconferencing system of claim 13 , the instructions further comprising instructions to:

detect image data indicative of a head within a field of view of the image capture device;

determine that the image data indicative of the head corresponds to a pixel height below a threshold corresponding to the boundary plane; and

exclude the image data indicative of the head from transmission to a remote endpoint as part of a visual data stream.

16. The videoconferencing system of claim 13 , the instructions further comprising instructions to:

detect image data indicative of a head within the polygon;

determine that the image data indicative of the head corresponds to a pixel height exceeding a threshold corresponding to the boundary plane; and

determine, responsive to determining that the image data indicative of the head corresponds to a pixel height exceeding the threshold corresponding to the boundary plane, that the image data indicative of the head corresponds to image data indicative of a face.

17. A method of operating a videoconferencing endpoint, comprising:

capturing a first image data frame using an image capture device of a videoconferencing endpoint, the first image data frame depicting a scene at the videoconferencing endpoint;

detecting image data indicative of a head within a polygonal region of the first image data frame;

determining that the image data indicative of the head corresponds to a pixel height below a threshold corresponding to the polygonal region;

excluding the image data indicative of the head from transmission to a remote endpoint as part of a visual data stream;

detecting second image data indicative of a second head within the polygonal region;

determining that the second image data indicative of the second head corresponds to a pixel height meeting the threshold; and

determining, responsive to determining that the image data indicative of the head corresponds to the pixel height meeting, that the second image data indicative of the second head corresponds to image data indicative of a face;

designating a plane as a boundary plane based, at least in part, on the visual patterns of the first image data frame;

displaying a depiction of the three-dimensional model using an electronic display device, the depiction of the three-dimensional model depicting the boundary plane as a selectable polygon; and

designating, based on a selection of the selectable polygon, a region which is behind the boundary plane as a region of exclusion, whereby image data emanating from the region of exclusion is excluded from consideration as being indicative of a meeting participant.

18. A method of operating a videoconferencing endpoint, comprising:

capturing a first image data frame using an image capture device of a videoconferencing endpoint, the first image data frame depicting a scene at the videoconferencing endpoint;

detecting image data indicative of a head within a polygonal region of the first image data frame;

determining that the image data indicative of the head corresponds to a pixel height below a threshold corresponding to the polygonal region;

excluding the image data indicative of the head from transmission to a remote endpoint as part of a visual data stream;

wherein the polygonal region corresponds to a planar boundary at a predetermined distance from the image capture device, and further comprising:

detecting a meeting participant at a first location at the videoconferencing endpoint;

framing the meeting participant at the first location within a first frame;

including data of the first frame within a visual stream for transmission to a remote endpoint;

tracking the meeting participant to a second location at the videoconferencing endpoint, the second location closer to the image capture device than the planar boundary;

framing the meeting participant at the second location within a second frame;

including data of the second frame corresponding to the meeting participant within the visual stream; and

excluding data of the second frame not corresponding to the meeting participant from the visual stream.

19. The method of claim 18 , wherein excluding data of the second frame not corresponding to the meeting participant comprises replacing the data of the second frame not corresponding to the meeting participant with other visual data.

Assignments (4)
NUNC PRO TUNC ASSIGNMENT Recorded Nov 13, 2023
From: PLANTRONICS, INC.
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 065549/0065 →
RELEASE OF PATENT SECURITY INTERESTS Recorded Aug 30, 2022
From: WELLS FARGO BANK, NATIONAL ASSOCIATION
To: PLANTRONICS, INC.; POLYCOM, INC.
Reel/Frame 061356/0366 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Mar 15, 2022
From: PLANTRONICS, INC.; POLYCOM, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION
Reel/Frame 059365/0413 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 23, 2021
From: ZHANG, KUI; KULKARNI, VARUN; KRISHNAMURTHY, RAGHAVENDRA BALAVALIKAR; BHATT, RAJEN; SCHAEFER, STEPHEN
To: PLANTRONICS, INC.
Reel/Frame 058468/0716 →
Continuity (1)
Related Publication 20230171300A1 · Jun 1, 2023