Inpainting dispatch techniques for digital images
Inpainting dispatch techniques for digital images are described. In one or more examples, an inpainting system includes a plurality of inpainting modules. The inpainting modules are configured to employ a variety of different techniques, respectively, as part of performing an inpainting operation. An inpainting dispatch module is also included as part of the inpainting system that is configured to select which of the plurality of inpainting modules are to be used to perform an inpainting operation for one or more regions in a digital image, automatically and without user intervention.
1 . A method comprising:
receiving, by a processing device, a digital image having one or more regions that are identified for use as part of an inpainting operation including merged regions generated by merging connected regions of the digital image;
selecting, by the processing device, a first inpainting module from a plurality of inpainting modules based on an amount of structure detected in an area of the digital image disposed proximal to the one or more regions;
receiving, by the processing device, the digital image as having an inpainting region generated by the inpainting operation applied to the one or more regions as performed by the first inpainting module;
responsive to detecting that the inpainting region generated by the first inpainting module includes an impermissible amount of artifacts based on a threshold, selecting a second inpainting module from the plurality of inpainting modules that is different than the first inpainting module; and
displaying, by the processing device, the digital image in a user interface including by regenerating the inpainting region by applying a second inpainting operation to the one or more regions as performed by the second inpainting module.
2 . The method as described in claim 1 , wherein the inpainting operation is configured to generate color values for pixels in the one or more regions.
3 . The method as described in claim 1 , wherein the selecting based on the amount of structure an amount of variability of pixels in the area of the digital image.
4 . The method as described in claim 1 , wherein the selecting the first inpainting module is based on a size of the one or more regions.
5 . The method as described in claim 1 , wherein the selecting the first inpainting module is performed using a machine-learning model.
6 . The method as described in claim 1 , wherein the selecting the first inpainting module is based on a determination that balances an amount of computational resource consumption by the plurality of inpainting modules, respectively, and quality of the inpainting region generated by the plurality of inpainting modules, respectively.
7 . The method as described in claim 1 , wherein the selecting includes:
generating labeled regions by labelling the connected regions;
generating the merged regions by merging the labeled regions;
generating classified regions by classifying the merged regions for use by a respective inpainting module of the plurality of inpainting modules; and
outputting an inpainting selection as specifying the classified regions.
8 . The method as described in claim 7 , further comprising optimizing the classified regions by merging at least two of the classified regions based on respective classifications.
9 . The method as described in claim 1 , wherein the plurality of inpainting modules include a patch-based inpainting module, a generative adversarial network (GAN) inpainting module, or a diffusion-based inpainting module.
10 . The method as described in claim 1 , wherein the impermissible amount of artifacts is based on at least one of a number of artifacts, an artifact density, or a size of artifacts in the inpainting region.
11 . An inpainting system comprising:
an inpainting dispatch module implemented by a processing device to generate an inpainting selection referencing a first inpainting module, the generating including selecting the first inpainting module from a plurality of inpainting modules using a machine-learning model based on result accuracy; and
the first inpainting module implemented by the processing device to perform an inpainting operation, based on the inpainting selection, to generate an inpainting region within one or more regions of a digital image by merging connected regions of the digital image; and
responsive to detecting that the inpainting region generated by the first inpainting module includes an impermissible amount of artifacts based on a threshold, the inpainting dispatch module implemented by the processing device to select a second inpainting module from the plurality of inpainting modules that is different than the first inpainting module and regenerate the inpainting region by applying a second inpainting operation to the one or more regions.
12 . The inpainting system as described in claim 11 , wherein the inpainting dispatch module is configured to select the first inpainting module based on an amount of structure detected in an area of the digital image disposed proximal to the one or more regions.
13 . The inpainting system as described in claim 11 , wherein the inpainting dispatch module is configured to select the first inpainting module based on a size of the one or more regions.
14 . The inpainting system as described in claim 11 , wherein the inpainting dispatch module is configured to select the first inpainting module using the machine-learning model as balancing the result accuracy with an amount of time to perform the inpainting operation, respectively.
15 . A computing device comprising:
a processing device; and
a computer-readable storage medium storing instructions that, responsive to execution by the processing device, causes the processing device to perform operations including:
receiving a digital image having one or more regions that are each identified for use as part of an inpainting operation including merged regions generated by merging connected regions of the digital image;
selecting a first inpainting module from a plurality of inpainting modules based on a size of the one or more regions;
receiving the digital image as having an inpainting region generated by the inpainting operation applied to the one or more regions as performed by the first inpainting module;
responsive to detecting that the inpainting region generated by the first inpainting module includes an impermissible amount of artifacts based on a threshold, selecting a second inpainting module from the plurality of inpainting modules that is different than the first inpainting module; and
displaying the digital image in a user interface including by regenerating the inpainting region by applying a second inpainting operation to the one or more regions as performed by the second inpainting module.
16 . The computing device as described in claim 15 , wherein the selecting the first inpainting module is further based on an amount of structure detected in an area of the digital image disposed proximal to the one or more regions.
17 . The computing device as described in claim 15 , wherein the selecting the first inpainting module is further based on a determination that balances an amount of computational resource consumption by the plurality of inpainting modules, respectively, and quality of the inpainting region generated by the plurality of inpainting modules, respectively.
18 . The computing device as described in claim 15 , wherein the selecting includes:
generating labeled regions by labelling the connected regions of the one or more regions;
generating the merged regions by merging the labeled regions;
generating classified regions by classifying the merged regions for use by a respective inpainting module of the plurality of inpainting modules; and
outputting an inpainting selection as specifying the classified regions.
19 . The computing device as described in claim 18 , further comprising optimizing the classified regions by merging at least two of the classified regions based on respective classifications.
20 . The computing device as described in claim 15 , wherein the impermissible amount of artifacts is based on at least one of a number of artifacts, an artifact density, or a size of artifacts in the inpainting region.