IP Library Granted Patent US 11,720,315
Granted Patent B1
US 11,720,315 · App. 17/877,915 · Granted Aug 8, 2023

Multi-stream video encoding for screen sharing within a communications session

Inventors: Zhichu He (Hangzhou, CN); Yi Guo (Hangzhou, CN); Bo Ling (Saratoga, CA); Jing Wu (Hangzhou, CN); Yichen Zhang (Hangzhou, CN)
Assignee: Zoom Video Communications, Inc.
G06F3/1454H04N7/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,720,315
App. No.
17/877,915
Granted
Aug 8, 2023
Kind
B1
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media related to multi-stream video encoding for screen sharing a communications session. The system may determine a sub-area and a remaining area of a display region. A first video stream of the sub-area is generated at a first frame rate. A second video stream of the remaining area is generated at second frame rate, where the second frame rate is a frame rate lower than the first frame rate. A client device may transmit the first video stream and the second video stream to a second client device.

Claims (65)

1. A computer-implemented method comprising:

determining a sub-area of a display region, the display region including a group of pixels and the sub-area including a subset of the group of pixels;

determining a remaining area of the display region, the remaining area including a subset of pixels from the group of pixels that are outside of the sub-area;

generating a first video stream of at least a portion of the sub-area of the display region, the first video stream having a first frame rate;

generating a second video stream of at least a portion of the remaining area of the display region, the second video stream having a second frame rate that is a frame rate lower than the first frame rate;

transmitting, by a first client device, the first video stream and the second video stream to a second client device;

determining a second sub-area of the display region;

generating a third video stream of the second sub-area, the third video stream having a third frame rate; and

transmitting, by the first client device, the third video stream to a second client device concurrently with the first video stream and the second video stream.

2. The computer-implemented method of claim 1 , wherein the sub-area of the display region includes an area of changing pixel values among consecutive frames of the display region.

3. The computer-implemented method of claim 1 , wherein the remaining area of the display region includes a portion of the display region that has a group of static pixel values among consecutive frames of the display region.

4. The computer-implemented method of claim 1 , further comprising:

determining whether pixels about a boundary of the sub-area are changing among frames of the display region; and

increasing a size of the sub-area to include those pixels that are determined to be changing in values, or decreasing a size of the sub-area to exclude those pixels that are determined to be not-change in values.

5. The computer-implemented method of claim 1 , wherein the first video stream is transmitted at a variable bitrate, and the second video stream is transmitted at a constant bitrate.

6. A non-transitory computer readable medium that stores executable program instructions that when executed by one or more computing devices configure the one or more computing devices to perform operations comprising:

determining a sub-area of a display region, the display region including a group of pixels and the sub-area including a subset of the group of pixels;

determining a remaining area of the display region, the remaining area including a subset of pixels from the group of pixels that are outside of the sub-area;

generating a first video stream of at least a portion of the sub-area of the display region, the first video stream having a first frame rate;

generating a second video stream of at least a portion of the remaining area of the display region, the second video stream having a second frame rate that is a frame rate lower than the first frame rate;

transmitting, by a first client device, the first video stream and the second video stream to a second client device;

determining a second sub-area of the display region;

generating a third video stream of the second sub-area, the third video stream having a third frame rate; and

transmitting, by the first client device, the third video stream to a second client device concurrently with the first video stream and the second video stream.

7. The non-transitory computer readable medium of claim 6 , wherein the sub-area of the display region includes an area of changing pixel values among consecutive frames of the display region.

8. The non-transitory computer readable medium of claim 6 , wherein the remaining area of the display region includes a portion of the display region that has a group of static pixel values among consecutive frames of the display region.

9. The non-transitory computer readable medium of claim 6 , the operations further comprising:

determining whether pixels about a boundary of the sub-area are changing among frames of the display region; and

increasing a size of the sub-area to include those pixels that are determined to be changing in values, or decreasing a size of the sub-area to exclude those pixels that are determined to be not-change in values.

10. The non-transitory computer readable medium of claim 6 , wherein the first video stream is transmitted at a variable bitrate, and the second video stream is transmitted at a constant bitrate.

11. A system comprising one or more processors configured to perform the operations of:

determining a sub-area of a display region, the display region including a group of pixels and the sub-area including a subset of the group of pixels;

determining a remaining area of the display region, the remaining area including a subset of pixels from the group of pixels that are outside of the sub-area;

generating a first video stream of at least a portion of the sub-area of the display region, the first video stream having a first frame rate;

generating a second video stream of at least a portion of the remaining area of the display region, the second video stream having a second frame rate that is a frame rate lower than the first frame rate;

transmitting, by a first client device, the first video stream and the second video stream to a second client device;

determining a second sub-area of the display region;

generating a third video stream of the second sub-area, the third video stream having a third frame rate; and

transmitting, by the first client device, the third video stream to a second client device concurrently with the first video stream and the second video stream.

12. The system of claim 11 , wherein the sub-area of the display region includes an area of changing pixel values among consecutive frames of the display region.

13. The system of claim 11 , wherein the remaining area of the display region includes a portion of the display region that has a group of static pixel values among consecutive frames of the display region.

14. The system of claim 11 , the operations further comprising:

determining whether pixels about a boundary of the sub-area are changing among frames of the display region; and

increasing a size of the sub-area to include those pixels that are determined to be changing in values, or decreasing a size of the sub-area to exclude those pixels that are determined to be not-change in values.

15. The system of claim 11 , wherein the first video stream is transmitted at a variable bitrate, and the second video stream is transmitted at a constant bitrate.

16. A computer-implemented method comprising:

determining a sub-area of a display region, the display region including a group of pixels and the sub-area including a subset of the group of pixels;

determining a remaining area of the display region, the remaining area including a subset of pixels from the group of pixels that are outside of the sub-area;

generating a first video stream of at least a portion of the sub-area of the display region, the first video stream having a first frame rate;

generating a second video stream of at least a portion of the remaining area of the display region, the second video stream having a second frame rate that is a frame rate lower than the first frame rate;

transmitting, by a first client device, the first video stream and the second video stream to a second client device;

receiving, by the first client device, a third video stream having a third frame rate, the third video stream generated by the second client device;

receiving, by the first client device, a fourth video stream having a fourth frame rate, the fourth frame rate being a frame rate greater than the third frame rate;

combining, by the first client device, the third video stream and the fourth video stream to form a composite video stream; and

providing for display the composite video stream to a user interface of the first client device, wherein the composite video stream displays at the frame rate of the third video stream.

17. A non-transitory computer readable medium that stores executable program instructions that when executed by one or more computing devices configure the one or more computing devices to perform operations comprising:

determining a sub-area of a display region, the display region including a group of pixels and the sub-area including a subset of the group of pixels;

determining a remaining area of the display region, the remaining area including a subset of pixels from the group of pixels that are outside of the sub-area;

generating a first video stream of at least a portion of the sub-area of the display region, the first video stream having a first frame rate;

generating a second video stream of at least a portion of the remaining area of the display region, the second video stream having a second frame rate that is a frame rate lower than the first frame rate;

transmitting, by a first client device, the first video stream and the second video stream to a second client device;

receiving, by the first client device, a third video stream having a third frame rate, the third video stream generated by the second client device;

receiving, by the first client device, a fourth video stream having a fourth frame rate, the fourth frame rate being a frame rate greater than the third frame rate;

combining, by the first client device, the third video stream and the fourth video stream to form a composite video stream; and

providing for display the composite video stream to a user interface of the first client device, wherein the composite video stream displays at the frame rate of the third video stream.

Assignments (2)
CHANGE OF NAME Recorded Jan 7, 2025
From: ZOOM VIDEO COMMUNICATIONS, INC.
To: ZOOM COMMUNICATIONS, INC.
Reel/Frame 069839/0593 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 7, 2022
From: ZHANG, YICHEN; HE, ZHICHU; GUO, YI; LING, BO; WU, JING
To: ZOOM VIDEO COMMUNICATIONS, INC.
Reel/Frame 061013/0017 →
Cited By (1)
US 12,687,997