Systems and methods for manipulation of shadows on portrait image frames
Systems and methods described herein may relate to potential methods of training a machine learning model to be implemented on a mobile computing device configured to capture, adjust, and/or store image frames. An example method includes supplying a first image frame of a subject in a setting lit within a first lighting environment and supplying a second image frame of the subject lit within a second lighting environment. The method further includes determining a mask. Additionally, the method includes combining the first image frame and the second image frame according to the mask to generate a synthetic image and assigning a score to the synthetic image. The method also includes training a machine learning model based on the assigned score to adjust a captured image based on the synthetic image.
1. A method, comprising:
causing an image capture device to capture an image frame;
comparing a shadow shape in the image frame to a plurality of synthetically shadowed image frames provided by a trained machine learning model;
based on the comparison, selecting a mask shape from a plurality of masks associated with the trained machine learning model, wherein the shape of the selected mask is one that most closely fits the shadow shape in the image frame; and
adjusting the image frame according to the mask shape to provide an adjusted image frame.
2. The method of claim 1 , wherein causing the image capture device to capture an image frame comprises capturing the image frame with a camera, opening an image frame file, or accessing the image frame by way of a cloud-based computing device.
3. The method of claim 1 , wherein the mask shape substantially matches a shape of at least a portion of a shadow within the image frame.
4. The method of claim 1 , wherein the adjusted image frame has more or less of a shadow than the image frame.
5. The method of claim 4 , wherein an image adjustment value is selected so as to soften, sharpen, or remove the shadow within the image frame.
6. The method of claim 5 , wherein image adjustment value comprises a noise value, wherein increasing the noise value will soften the shadow within the image frame and decreasing the noise value will sharpen the shadow within the image frame.
7. A system, comprising:
a computing device including a processor and a non-transitory computer readable medium wherein the non-transitory computer readable medium stores a set of program instructions provided by a trained machine learning model, wherein the processor executes the program instructions so as to carry out operations, the operations comprising:
causing an image capture device to capture an image frame;
comparing a shadow shape in the image frame to a plurality of synthetically shadowed image frames provided by the trained machine learning model;
based on the comparison, selecting a mask shape from a plurality of masks associated with the trained machine learning model, wherein the shape of the selected mask is one that most closely fits the shadow shape in the image frame;
adjusting the image frame according to the mask shape to provide an adjusted image frame; and
displaying the adjusted image frame.
8. The system of claim 7 , wherein the computing device comprises at least one of: a mobile computing device, a laptop, a cloud-based computing device, or a desktop computer.
9. The system of claim 7 , wherein causing the image capture device to capture an image frame comprises capturing the image frame with a camera, opening an image frame file, or accessing the image frame by way of a cloud-based computing device.
10. The system of claim 7 , further comprising a graphical user interface, wherein the graphical user interface comprises a control interface configured to controllably adjust the image frame according to the selected mask.
11. The system of claim 10 , wherein the control interface comprises tuning knobs that are controllably operable to soften, sharpen, or eliminate shadows within the image frame.
12. The system of claim 7 , wherein adjusting the image frame according to the mask shape is done automatically.