Modifying shadows in image data
Systems and techniques are described herein for modifying images. For instance, a method for modifying images is provided. The method may include receiving an image, the image including a first plurality of pixels representing a first shadow of a first shadow type and a second plurality of pixels representing a second shadow of a second shadow type; processing the image to detect the first shadow; and modifying the first plurality of pixels of the image to generate an output image, wherein the output image includes the second plurality of pixels representing the second shadow.
1 . An apparatus for modifying images, the apparatus comprising:
at least one memory; and
at least one processor coupled to the at least one memory and configured to:
receive an image, the image including a first plurality of pixels representing a first shadow of a first shadow type, a second plurality of pixels representing a second shadow of a second shadow type, and a third plurality of pixels representing the first shadow and the second shadow, wherein the first shadow type is cast by a first type of object, wherein the second shadow type is cast by a second type of object, and wherein the first type of object is different than the second type of object;
process the image to detect the first shadow; and
modify the first plurality of pixels and the third plurality of pixels of the image to generate an output image, wherein the output image includes the second plurality of pixels representing the second shadow and wherein the third plurality of pixels are modified in the output image to reduce the first shadow in the third plurality of pixels.
2 . The apparatus of claim 1 , wherein the first type of object comprises at least one of:
a hand of a person operating a handheld device to capture the image;
the handheld device used to capture the image; or
a head of the person.
3 . The apparatus of claim 1 , wherein the third plurality of pixels are modified to preserve the second shadow.
4 . The apparatus of claim 1 , wherein the first shadow is detected based on a user input.
5 . The apparatus of claim 1 , wherein the at least one processor is configured to determine that an image is a candidate for shadow removal based on image-capture conditions related to the image.
6 . The apparatus of claim 5 , wherein the image-capture conditions comprise at least one of:
a focus setting used to capture the image;
an exposure setting used to capture the image;
a tilt angle of a device used to capture the image;
a subject of the image being stationary; or
a classification of the image by an image classifier.
7 . The apparatus of claim 5 , wherein the image is determined to be a candidate for shadow removal using a machine-learning model trained to identify candidate images for shadow removal based on at least one of:
a focus setting used to capture of the image;
an exposure setting used to capture the image;
a tilt angle of a device used to capture the image; or
a subject of the image being stationary.
8 . The apparatus of claim 5 , wherein the image is determined to be a candidate for shadow removal by a machine-learning model trained to identify shadows of the first shadow type in images.
9 . The apparatus of claim 1 , wherein the first plurality of pixels of the image are determined using a machine-learning model trained to identify shadows of the first shadow type in images.
10 . The apparatus of claim 1 , wherein the first plurality of pixels of the image are modified using a machine-learning model trained to remove shadows of the first shadow type from images.
11 . The apparatus of claim 1 , wherein the at least one processor is configured to:
analyze a light source that resulted in the first shadow; and
modify, based on analyzing the light source, the image to simulate a change to the light source.
12 . The apparatus of claim 11 , wherein the image comprises a first image captured by a device at a first position, wherein the light source is analyzed based on:
a second image captured by the device at a second position; and
a third plurality of pixels that represent the first shadow in the second image.
13 . The apparatus of claim 11 , wherein the image is modified using a tone-mapping algorithm.
14 . The apparatus of claim 11 , wherein the image is modified to simulate a change to at least one of:
a position of the light source;
an intensity of the light source; or
a hue of the light source.
15 . A method for modifying images, the method comprising:
receiving an image, the image including a first plurality of pixels representing a first shadow of a first shadow type, a second plurality of pixels representing a second shadow of a second shadow type, and a third plurality of pixels representing the first shadow and the second shadow, wherein the first shadow type is cast by a first type of object, wherein the second shadow type is cast by a second type of object, and wherein the first type of object is different than the second type of object;
processing the image to detect the first shadow; and
modifying the first plurality of pixels and the third plurality of pixels of the image to generate an output image, wherein the output image includes the second plurality of pixels representing the second shadow and wherein the third plurality of pixels are modified in the output image to reduce the first shadow in the third plurality of pixels.
16 . The method of claim 15 , wherein the first type of object comprises at least one of:
a hand of a person operating a handheld device to capture the image;
the handheld device used to capture the image; or
a head of the person.
17 . The method of claim 15 , wherein the third plurality of pixels are modified to preserve the second shadow.
18 . The method of claim 15 , wherein the first shadow is detected based on a user input.