Dynamic lighting level adjustment for virtual backgrounds
A system can automatically adjust a virtual background used by a participant in a video conference to match lighting level conditions present in a foreground of a participant video stream captured at a participant device. The matching can be done by adjusting one or more virtual background parameters, including but not limited to, exposure, brightness, contrast, and ISO. In some implementations, the system can dynamically or automatically adjust the lighting levels of the virtual background based on changes that may occur to the lighting level of the foreground during the video conference.
1 . A method, comprising:
detecting, by a first device, a participant in a video stream;
determining, by the first device, a lighting directionality for the participant based on exposure information from pixels in a pixel distribution proximate to a virtual background used by the participant during the video stream;
adjusting, automatically by the first device, the virtual background based on the lighting directionality to generate an adjusted virtual background; and
outputting, by the first device, an adjusted video stream including the adjusted virtual background for display at a second device.
2 . The method of claim 1 , wherein determining a lighting directionality for the participant based on a pixel distribution of the participant in the video stream further comprising:
determining a light pixel distribution of the participant in the video stream.
3 . The method of claim 1 , wherein determining a lighting directionality for the participant based on a pixel distribution of the participant in the video stream further comprising:
determining a dark pixel distribution of the participant in the video stream.
4 . The method of claim 1 , wherein determining a lighting directionality for the participant based on a pixel distribution of the participant in the video stream further comprising:
determining a light pixel distribution and a dark pixel distribution of the participant in the video stream.
5 . The method of claim 1 , wherein determining a lighting directionality for the participant based on a pixel distribution of the participant in the video stream further comprising:
determining a bright pixel distribution and a dark pixel distribution of the participant in the video stream.
6 . The method of claim 1 , wherein determining a lighting directionality for the participant based on a pixel distribution of the participant in the video stream further comprising:
determining a lighting level pixel distribution and a dark pixel distribution of the participant in the video stream.
7 . The method of claim 1 , wherein determining a lighting directionality for the participant based on a pixel distribution of the participant in the video stream further comprising:
determining a spatial pixel distribution and a dark pixel distribution of the participant in the video stream.
8 . The method of claim 1 , further comprising:
dynamically applying the lighting directionality to the virtual background during the video stream.
9 . An apparatus, comprising:
a memory; and
a processor configured to execute instructions stored in the memory to:
detect a participant in a video stream;
determine a lighting directionality for the participant based on exposure information from pixels in a pixel distribution proximate to a virtual background used by the participant during the video stream;
adjust, automatically, the virtual background based on the lighting directionality to generate an adjusted virtual background; and
output an adjusted video stream including the adjusted virtual background for display at a device.
10 . The apparatus of claim 9 , wherein the processor is configured to execute the instructions to:
generate the pixel distribution from a spatial distribution of light pixels.
11 . The apparatus of claim 9 , wherein the processor is configured to execute the instructions to:
generate the pixel distribution from a spatial distribution of dark pixels.
12 . The apparatus of claim 9 , wherein the processor is configured to execute the instructions to:
generate the pixel distribution from a spatial distribution of dark pixels and light pixels.
13 . The apparatus of claim 9 , wherein the processor is configured to execute the instructions to:
generate the pixel distribution from a spatial distribution of lighting levels of pixels.
14 . The apparatus of claim 9 , wherein the processor is configured to execute the instructions to:
provide a graded or shaded adjustment to the virtual background by application of the lighting directionality.
15 . A non-transitory computer-readable medium storing instructions operable to cause one or more processors to perform operations comprising:
detecting, by a first device, a participant in a video stream;
determining, by the first device, a lighting directionality for the participant based on exposure information from pixels in a pixel distribution proximate to a virtual background used by the participant during the video stream;
adjusting, automatically by the first device, the virtual background based on the lighting directionality to generate an adjusted virtual background; and
outputting, by the first device, an adjusted video stream including the adjusted virtual background for display at a second device.
16 . The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to execute the instructions to perform operations comprising:
initiating detection of the participant upon receiving an enablement notification.
17 . The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to execute the instructions to perform operations comprising:
generating the pixel distribution from a spatial distribution of pixel lighting levels.
18 . The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to execute the instructions to perform operations comprising:
providing a seamless transition between the participant and the virtual background by application of the lighting directionality.
19 . The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to execute the instructions to perform operations comprising:
generating the pixel distribution by comparing which pixels are light and which pixels are dark.
20 . The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to execute the instructions to perform operations comprising:
generating the pixel distribution by comparing which regions are light and which regions are dark.