Techniques for smooth region merging in image editing

1. A computer-implemented method comprising:

cropping, by a computing system, an input image around a region to be edited to produce a cropped input image;

applying, by the computing system, an affine transformation to the cropped input image to produce a rotated cropped input image;

providing, by the computing system, the rotated cropped input image as input to a machine learning model to generate a latent space representation of the rotated cropped input image;

editing, by the computing system, the latent space representation to generate an edited latent space representation;

providing, by the computing system, the edited latent space representation as input to a trained generator neural network implemented by the computing system;

generating, by the generator neural network, a generated edited image;

applying, by the computing system, an inverse affine transformation to the generated edited image to generate a rotated generated edited image;

aligning, by the computing system, an identified segment of the rotated generated edited image with an identified corresponding segment of the input image to produce an aligned rotated generated edited image; and

blending, by the computing system, the aligned rotated generated edited image with the input image to generate an edited output image.

2. The method of claim 1 , further comprising:

identifying, by the computing system, the segment of the rotated generated edited image; and

identifying, by the computing system, the corresponding segment of the input image.

3. The method of claim 2 , wherein identifying the corresponding segment of the input image comprises:

providing, by the computing system, the input image to a segmentation neural network implemented by the computing system, wherein the segmentation neural network identifies a plurality of segments including the segment of the input image.

4. The method of claim 1 , further comprising:

based on the identified corresponding segment of the input image and the identified segment of the rotated generated edited image, modifying, by the computing system, a color and a contrast in the rotated generated edited image to match a color and a contrast in the input image.

5. The method of claim 1 , wherein blending the aligned rotated generated edited image with the input image comprises applying, by the computing system, healing to the aligned rotated generated edited image and the input image.

6. The method of claim 1 , further comprising:

identifying, by the computing system, pixels corresponding to artifacts in the aligned rotated generated edited image; and

applying a content-aware fill to the identified pixels of the aligned rotated generated edited image.

7. The method of claim 1 , further comprising outputting, by the computing system, the edited output image to a display device for display.

8. The method of claim 1 , further comprising, before cropping the input image, detecting, by the computing system, a target region of the input image for configuring the cropping.

9. A computing system comprising:

a processor;

a non-transitory computer-readable medium comprising instructions which, when executed by the processor, perform processing comprising:

applying an affine transformation to an input image to produce a rotated input image;

providing the rotated input image as input to a machine learning model to generate a latent space representation of the rotated input image;

editing the latent space representation to generate an edited latent space representation;

providing the edited latent space representation as input to a trained generator neural network implemented by the computing system;

generating, by the generator neural network, an edited generated image;

applying an inverse affine transformation to the generated edited image to generate a rotated generated edited image;

aligning an identified segment of the rotated generated edited image with an identified corresponding segment of the input image to produce an aligned rotated generated edited image; and

blending the aligned rotated generated edited image with the input image to generate an edited output image.

10. The computing system of claim 9 , the processing further comprising:

identifying the segment of the rotated generated edited image; and

identifying the corresponding segment of the input image.

11. The computing system of claim 10 , wherein identifying the corresponding segment of the input image comprises:

providing, by the computing system, the input image to a segmentation neural network implemented by the computing system, wherein the segmentation neural network identifies a plurality of segments including the segment of the input image.

12. The computing system of claim 9 , the processing further comprising:

outputting the edited output image for display.

13. The computing system of claim 9 , the processing further comprising:

based on the identified corresponding segment of the input image and the identified segment of the rotated generated edited image, adjusting, by the computing system, a color and a contrast in the rotated generated edited image to match a color and a contrast in the input image.

14. The computing system of claim 9 , wherein blending the aligned rotated generated edited image with the input image comprises applying, by the computing system, healing to the aligned rotated generated edited image and the input image.

15. The computing system of claim 9 , the processing further comprising:

identifying, by the computing system, pixels corresponding to artifacts in the aligned rotated generated image; and

applying a content-aware fill to the identified pixels of the aligned rotated generated image.

16. A non-transitory computer-readable medium having instructions stored thereon, the instructions executable by a processing device to perform operations comprising:

cropping an input image around a region to be edited to produce a cropped input image;

applying an affine transformation to the cropped input image to produce a rotated cropped input image;

providing the rotated cropped input image as input to a machine learning model to generate a latent space representation of the rotated cropped input image;

editing the latent space representation to generate an edited latent space representation;

providing the edited latent space representation as input to a trained generator neural network;

generating, by the generator neural network, a generated edited image; and

a step for blending the generated edited image with the input image such that an identified segment of the generated edited image aligns with an identified corresponding segment of the input image.

17. The medium of claim 16 , the operations further comprising:

identifying the segment of the generated edited image; and

identifying the corresponding segment of the input image.

18. The medium of claim 17 , wherein identifying the corresponding segment of the input image comprises:

providing the input image to a segmentation neural network, wherein the segmentation neural network identifies a plurality of segments including the segment of the input image.

19. The medium of claim 16 , the operations further comprising:

outputting the edited output image for display.

20. The medium of claim 16 , the operations further comprising:

identifying, by the computing system, pixels corresponding to artifacts in the aligned rotated generated image; and

applying a content-aware fill to the identified pixels of the aligned rotated generated image.