Optimized facial illumination from adaptive screen content
Aspects of the present disclosure relate to adjusting an illumination of a user depicted in one or more images when using a video conferencing application. In one example, one or more images depicting the user may be received from an image sensor. Further, an illumination of the user depicted in the one or more images may be determined to be unsatisfactory. For example, the user's face may be too bright or too dim. Accordingly, content displayed at a display device may identified and then modified. The modified content may then be rendered to a display device thereby changing the illumination of the user depicted in subsequent images. In examples, the modified content may include a graphical element, such as a ring of a specific color at least partially surrounding content rendered to and displayed at the display device.
1. A method for adjusting an illumination of a user depicted in one or more images, the method comprising:
receiving first image data from an image sensor as content for display, wherein the first image data indicates the user;
determining, based on a feature associated with the first image data, respective values of two or more lighting parameters,
wherein the two or more lighting parameters comprise:
a first lighting parameter associated with enhancing the first image data,
a second lighting parameter associated with image sensor settings for subsequently capturing second image data indicating the user, and
a third lighting parameter associated with illuminating the user; and
adapting, based on the respective values of the two or more lighting parameters, the content for display.
2. The method of claim 1 , wherein adapting the content includes:
extracting the feature associated with the first image data;
generating a graphical element to render at a display device based on the feature associated with the first image data and the first image data; and
causing the graphical element to be rendered at the display device, the graphical element changing the illumination of the user.
3. The method of claim 2 , further comprising:
changing at least one of a position of the graphical element, a size of the graphical element, and a color of the graphical element.
4. The method of claim 2 , wherein the graphical element is at least one of a light ring, light square, and light bar.
5. The method of claim 1 , wherein the feature associated with the first image data includes a location associated with a face of the user, and wherein the adapted content for display is displayed at a display device such that the adapted content illuminates the face of the user.
6. The method of claim 1 , wherein adapting the content displayed for display further comprises:
receiving an image identifying content displayed at a display device;
rearranging the content; and
causing the rearranged content to be rendered at the display device, the rearranged content changing the illumination of the user.
7. The method of claim 6 , wherein rearranging the content further comprises:
identifying one or more application windows displayed at the display device; and
moving at least one application of the one or more application windows from a first position to a second position.
8. The method of claim 1 , further comprising:
generating a graphical element to render at a display device based on the feature associated with the first image data and the first image data; and
causing the graphical element to be rendered at the display device, the graphical element changing the illumination of the user.
9. The method of claim 1 , further comprising changing at least one configuration setting associated with the image sensor based on the determining that the illumination of the user depicted in the first image data is unsatisfactory.
10. The method of claim 1 , further comprising:
extracting the feature associated with the first image data;
providing the feature, an image of the received content currently displayed at a display device, and the first image data to a neural network model; and
generating, by the neural network model, updated image of the received content.
11. The method of claim 1 , further comprising:
providing the feature, an image of content currently displayed, and the first image data to a neural network model; and
generating, by the neural network model, the adapted content and updated image of the received content.
12. The method of claim 1 , further comprising:
providing the feature and the first image data to a neural network model; and
generating, by the neural network model, the adapted content.
13. A system for adjusting an illumination of a user depicted in one or more images, the system comprising:
a processor;
an image sensor; and
memory including instructions, which when executed by the processor, cause the processor to perform:
receiving first image data from the image sensor as content for display, wherein the first image data indicates the user;
determining, based on a feature associated with the first image data, respective values of two or more lighting parameters,
wherein the two or more lighting parameters comprise:
a first lighting parameter associated with enhancing the first image data,
a second lighting parameter associated with image sensor settings for subsequently capturing second image data indicating the user, and
a third lighting parameter associated with illuminating the user; and
adapting, based on the respective values of the two or more lighting parameters, the content for display, wherein the adapted content indicates the user.
14. The system of claim 13 , wherein the instructions, when executed by the processor, cause the processor to:
move at least one user interface element rendered to a display device based on the two or more lighting parameters.
15. The system of claim 13 , wherein the instructions, when executed by the processor, cause the processor to:
change at least one configuration parameter of the image sensor.
16. The system of claim 13 , wherein the instructions, when executed by the processor, cause the processor to:
generate a graphical element to render at a display device based on the two or more lighting parameters; and
cause the graphical element to be rendered at the display device, the graphical element changing the illumination of the user.
17. A device comprising a processor configured to execute a process comprising:
receiving first image data from an image sensor as content for display, wherein the first image data indicates a user;
determining, based on a feature associated with the first image data, respective values of two or more lighting parameters,
wherein the two or more lighting parameters comprise:
a first lighting parameter associated with enhancing the first image data,
a second lighting parameter associated with image sensor settings for subsequently capturing second image data indicating the user, and
a third lighting parameter associated with illuminating the user; and
adapting, based on the respective values of the two or more lighting parameters associated, the content for display, wherein the content indicates the user.
18. The device of claim 17 , wherein the processor further configured to execute the process comprising:
generating a graphical element to render at a display device; and
causing the graphical element to be rendered at the display device, the graphical element changing an illumination of the user.
19. The device of claim 17 , wherein the processor further configured to execute the process comprising:
changing an illumination of the user depicted in the first image data such that a brightness associated with a portion of the user depicted in the first image data is reduced.
20. The device of claim 17 , wherein the processor further configured to execute the process comprising:
receiving user input; and
determining that an illumination of the user depicted in the first image data is unsatisfactory based on the received user input.