IP Library Granted Patent US 11,895,438
Granted Patent B2
US 11,895,438 · App. 17/579,756 · Granted Feb 6, 2024

Virtual background adjustment based on conference participant lighting levels

Inventor: Thanh Le Nguyen (Belle Chasse, LA)
Assignee: Zoom Video Communications, Inc.
H04N7/15G06V10/141G06V10/60H04L12/1822H04N23/71H04N23/74
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,895,438
App. No.
17/579,756
Granted
Feb 6, 2024
Kind
B2
Abstract

A system can automatically adjust a virtual background used by a participant in a video conference to match lighting level conditions present in a foreground of a participant video stream captured at a participant device. The matching can be done by adjusting one or more virtual background parameters, including but not limited to, exposure, brightness, contrast, and ISO. In some implementations, the system can dynamically or automatically adjust the lighting levels of the virtual background based on changes that may occur to the lighting level of the foreground during the video conference.

Claims (63)

1. A method, comprising:

detecting a participant in a video stream during a video conference;

determining a lighting level for the participant;

comparing the lighting level for the participant in the video stream with a lighting level of a virtual background used by the participant during the video conference;

adjusting, based on the comparison of the lighting level for the participant with the lighting level of the virtual background, the lighting level of the virtual background;

determining a lighting directionality for the participant based on a foreground spatial distribution of the participant in the video stream;

applying the lighting directionality to the virtual background; and

outputting an adjusted video stream including the adjusted virtual background for rendering within a user interface.

2. The method of claim 1 , wherein the virtual background is dynamically adjusted based on one or more participant lighting levels during the video conference.

3. The method of claim 1 , further comprising:

adjusting the lighting level of the participant based on one or more video quality thresholds.

4. The method of claim 1 , further comprising:

adjusting the lighting level of the participant based on one or more video quality thresholds; and

adjusting the lighting level of the virtual background to match the adjusted lighting level of the participant.

5. The method of claim 1 , further comprising:

generating a participant histogram of the lighting level of the participant;

generating a virtual background histogram of the lighting level of the virtual background; and

comparing the participant histogram with the virtual background histogram to determine amount of lighting level adjustment.

6. The method of claim 1 , wherein the adjusting further comprising:

adjusting one or more virtual background parameters.

7. The method of claim 6 , wherein the one or more virtual background parameters includes exposure, brightness, contrast, and International Organization for Standardization level.

8. The method of claim 1 , wherein determining the lighting directionality for the participant based on the foreground spatial distribution comprises:

utilizing a portion of pixels represented in the foreground spatial distribution.

9. The method of claim 1 , further comprising:

setting one or more lighting levels of the virtual background using a machine learning model upon recognition of a participant configuration.

10. The method of claim 1 , further comprising:

adjusting the lighting level of the participant in the video stream and the adjusted virtual background based on one or more video quality thresholds.

11. The method of claim 1 , wherein the lighting level of the participant in the video stream is an average of lighting levels of pixels comprising the participant.

12. The method of claim 1 , wherein the lighting level of the participant in the video stream is an average of lighting levels of pixels comprising a border of the participant.

13. An apparatus, comprising:

a memory; and

a processor configured to execute instructions stored in the memory to:

detect a participant in a video stream during a video conference;

determine a lighting level for the participant;

compare the lighting level for the participant with a lighting level of a virtual background used by the participant during the video conference;

adjust, based on the comparison of the lighting level for the participant with the lighting level of a virtual background, the lighting level of the virtual background;

determine a lighting directionality for the participant based on a foreground spatial distribution of the participant in the video stream;

apply the lighting directionality to the virtual background; and

output an adjusted video stream including the adjusted virtual background for rendering within a user interface.

14. The apparatus of claim 13 , wherein the processor is configured to execute the instructions to:

generate a spatial distribution of the lighting level of the participant in the video stream; and

use the spatial distribution to adjust the virtual background to match the lighting level of the participant.

15. The apparatus of claim 13 , wherein the processor is configured to execute the instructions to:

determine virtual background regions proximate to the participant; and

adjust, based on the lighting level for the participant, lighting levels for the virtual background regions to match the lighting level of the participant proximate to each virtual background region.

16. The apparatus of claim 13 , wherein the processor is configured to execute the instructions to:

determine whether facial features of the participant are in shadow areas based on a spatial distribution of the lighting level of the participant in the video stream; and

adjust those portions of the facial features that are in the shadow areas.

17. A non-transitory computer readable medium storing instructions operable to cause one or more processors to perform operations comprising:

detecting a participant in a video stream during a video conference;

determining a lighting level for the participant;

comparing the lighting level for the participant with a lighting level of a virtual background used by the participant during the video conference;

adjusting, based on the comparison of the lighting level for the participant with the lighting level of the virtual background, the lighting level of the virtual background;

determining a lighting directionality for the participant based on a foreground spatial distribution of the participant in the video stream;

applying the lighting directionality to the virtual background; and

outputting an adjusted video stream including the adjusted virtual background for rendering within a user interface.

18. The non-transitory computer readable medium of claim 17 , wherein the processor is configured to execute the instructions to perform operations comprising:

initiating detection of the participant upon receiving an enablement notification.

19. The non-transitory computer readable medium of claim 17 , wherein the processor is configured to execute the instructions to perform operations comprising:

generating a participant histogram of the lighting level of the participant; and

setting the lighting level of the participant to a most frequent lighting level in the participant histogram.

20. The non-transitory computer readable medium of claim 17 , wherein the processor is configured to execute the instructions to perform operations comprising:

setting one or more lighting levels of the virtual background using a machine learning model upon recognition of a presentation configuration.

Assignments (2)
CHANGE OF NAME Recorded Jan 7, 2025
From: ZOOM VIDEO COMMUNICATIONS, INC.
To: ZOOM COMMUNICATIONS, INC.
Reel/Frame 069839/0593 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 20, 2022
From: NGUYEN, THANH LE
To: ZOOM VIDEO COMMUNICATIONS, INC.
Reel/Frame 058706/0747 →
Continuity (1)
Related Publication 20230231972A1 · Jul 20, 2023
Cited By (1)
US 12,689,708