Interactive system for automatically synthesizing a content-aware fill

1. A computerized method for automatically synthesizing a target image to fill a hole in an original image, the method comprising:

providing a user interface configured to:

receive a first user input specifying the hole in the original image to be filled, leaving a remaining portion of the original image; and

receive a second user input customizing a sampling region of the remaining portion of the original image from which pixels can be sampled;

storing the sampling region in a constraint mask; and

passing the constraint mask to a patch-based synthesizer configured to synthesize the target image from patches sampled from the sampling region specified by the constraint mask;

wherein the user interface is configured to present the original image with the synthesized target image filling the hole.

2. The method of claim 1 , wherein the user interface is configured to present the original image with an overlay indicating the sampling region of the original image.

3. The method of claim 1 , wherein the second user input comprises a brush tool input modifying the overlay to modify the sampling region.

4. The method of claim 1 , further comprising, upon detecting completion of the second user input, automatically updating the constraint mask and passing the updated constraint mask to the patch-based synthesizer.

5. The method of claim 1 , wherein the user interface is configured to present a gradually updating preview of what the target image will look like, based on the second user input, prior to completion of the target image.

6. The method of claim 5 , wherein the gradually updating preview is synthesized by successive iterations of the patch-based synthesizer.

7. The method of claim 1 , wherein the user interface is configured to receive a third user input specifying one or more similarity transform parameters for synthesizing the target image.

8. The method of claim 1 , wherein the user interface is configured to receive:

a fourth user input specifying a range over which the patches can rotate to synthesize the target image; and

a fifth user input specifying a color adaptation to change brightness and contrast of the patches to synthesize the target image.

9. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:

accessing a constraint mask specifying a customizable sampling region of an original image from which pixels can be sampled, the sampling region excluding a user-specified hole in the original image, the hole leaving a remaining portion of the original image, the customizable sampling region being a subset of the remaining portion;

synthesizing a target image to fill the hole from patches sampled from the sampling region specified by the constraint mask; and

providing the synthesized target image for presentation of the original image with the synthesized target image filling the hole.

10. The one or more computer storage media of claim 9 , the operations further comprising, upon completion of a user input modifying the sampling region, receiving an updated constraint mask corresponding to the modified sampling region, and synthesizing an updated target image from patches sampled from the modified sampling region specified by the updated constraint mask.

11. The one or more computer storage media of claim 10 , wherein synthesizing the updated target image comprises salvaging computations from a prior iteration.

12. The one or more computer storage media of claim 9 , the operations further comprising providing a preview of what the target image will look like prior to completion.

13. The one or more computer storage media of claim 12 , wherein the preview is a gradually updating preview of the target image synthesized by successive iterations of the patch-based synthesizer.

14. The one or more computer storage media of claim 9 , the operations further comprising receiving one or more user-specified similarity transform parameters, wherein the patches are generated using the user-specified similarity transform parameters.

15. The one or more computer storage media of claim 9 , the operations synthesizing the target image in a deterministic fill synthesis mode.

16. A computer system comprising:

one or more hardware processors and memory configured to provide computer program instructions to the one or more hardware processors;

an interface component configured to:

receive a first user input specifying a hole in an original image to be filled leaving a remaining portion of the original image; and

receive a second user input customizing a sampling region of the remaining portion of the original image from which pixels can be sampled, the sampling region excluding the hole; and

a means for synthesizing a gradually updating target image to fill the hole using patches sampled from the sampling region;

wherein the interface component is configured to present the original image with the gradually updating target image filling the hole.

17. The computer system of claim 16 , wherein the interface component is configured to present the original image with an overlay indicating the sampling region of the original image.

18. The computer system of claim 17 , wherein the second user input comprises a brush tool input modifying the overlay, and wherein the means for synthesizing a gradually updating target image is configured, upon completion of the brush tool input, to automatically update the target image based on the modified sampling region.

19. The computer system of claim 16 , wherein the interface component is configured to receive one or more user-specified similarity transform parameters, and wherein the means for synthesizing a gradually updating target image is configured to synthesize the target image from similarity transformed patches generated using the user-specified similarity transform parameters.

20. The computer system of claim 19 , wherein the one or more user-specified similarity transform parameters comprise a user selection of allowable rotations, scaling and mirroring.