IP Library Granted Patent US 10,586,311
Granted Patent B2
US 10,586,311 · App. 15/921,457 · Granted Mar 10, 2020

Patch validity test

Inventors: Sohrab Amirghodsi (Seattle, WA); Kevin Wampler (Seattle, WA); Elya Shechtman (Seattle, WA); Aliakbar Darabi (Newcastle, WA)
Assignee: Adobe Inc.
G06T5/005G06K9/6215G06T5/20G06T11/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,586,311
App. No.
15/921,457
Granted
Mar 10, 2020
Kind
B2
Abstract

Embodiments of the present invention provide systems, methods, and computer storage media for improved patch validity testing for patch-based synthesis applications using similarity transforms. The improved patch validity tests are used to validate (or invalidate) candidate patches as valid patches falling within a sampling region of a source image. The improved patch validity tests include a hole dilation test for patch validity, a no-dilation test for patch invalidity, and a comprehensive pixel test for patch invalidity. A fringe test for range invalidity can be used to identify pixels with an invalid range and invalidate corresponding candidate patches. The fringe test for range invalidity can be performed as a precursor to any or all of the improved patch validity tests. In this manner, validated candidate patches are used to automatically reconstruct a target image.

Claims (42)

1. A computerized method for patch validity testing, the method comprising:

generating a similarity transformed candidate patch for reconstructing a target image, wherein the similarity transformed candidate patch comprises pixels from a source image;

before using the similarity transformed candidate patch to reconstruct the target image, verifying that the similarity transformed candidate patch falls within a designated sampling region of the source image by performing one or more patch validity tests comprising an evaluation of a representative pixel of the similarity transformed candidate patch; and

automatically reconstructing the target image using the verified similarity transformed candidate patch.

2. The method of claim 1 , wherein automatically reconstructing the target image fills a hole in the source image, wherein a first of the one or more patch validity tests comprises:

dilating the hole, thereby generating a reduced sampling region; and

verifying the similarity transformed candidate patch falls within the designated sampling region based on performing a lookup to determine that the representative pixel falls within the reduced sampling region.

3. The method of claim 1 , wherein dilating the hole comprises adding a dilation strip with a width based on width of the similarity transformed candidate patch, allowed patch rotations, a maximum allowed scaling factor, and a super sampling rate.

4. The method of claim 1 , wherein the representative pixel is a center pixel of the similarity transformed candidate patch.

5. The method of claim 1 , wherein a second of the one or more patch validity tests comprises verifying the similarity transformed candidate patch falls within the designated sampling region based on performing a lookup to determine whether the representative pixel falls outside of the sampling region.

6. The method of claim 5 , wherein the representative pixel is a center pixel of the similarity transformed candidate patch.

7. The method of claim 1 , wherein a third of the patch validity tests comprise verifying the similarity transformed candidate patch falls within the designated sampling region based on performing a lookup for each of the pixels of the similarity transformed candidate patch to determine that each pixel falls within the sampling region, wherein performing a lookup for each of the pixels of the similarity transformed candidate patch comprises calculating pixel coordinates of each pixel in a patch row using one fused multiple-add vector instruction.

8. The method of claim 1 , wherein performing the one or more patch validity tests includes executing a fringe test for range invalidity comprising:

adding a fringe to the source image; and

verifying the similarity transformed candidate patch based on performing a lookup to determine whether one of the pixels of the similarity transformed candidate patch falls within the fringe.

9. The method of claim 1 , wherein performing the one or more patch validity tests includes executing the fringe test for range invalidity as a precursor to each of the one or more patch validity tests.

10. The method of claim 8 , wherein the fringe has a width based on width of the similarity transformed candidate patch, allowed patch rotations, a maximum allowed scaling factor, and a super sampling rate.

11. One or more non-transitory computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:

generating a similarity transformed candidate patch for reconstructing a target image, wherein the similarity transformed candidate patch comprises pixels from a source image; and

before using the similarity transformed candidate patch to reconstruct the target image, verifying that the similarity transformed candidate patch falls within a designated sampling region of the source image by performing one or more patch validity tests comprising at least one of:

a first patch validity test comprising:

reducing the sampling region; and

verifying the similarity transformed candidate patch falls within the designated sampling region based on performing a lookup to determine that a representative pixel of the similarity transformed candidate patch falls within the reduced sampling region;

a second patch validity test comprising verifying the candidate patch falls within the designated sampling region based on performing a lookup to determine whether the representative of the similarity transformed candidate patch pixel falls outside of the sampling region; or

a third patch validity test comprising verifying the similarity transformed candidate patch falls within the designated sampling region based on performing a lookup for each of the pixels of the similarity transformed candidate patch to determine that each pixel falls within the sampling region, wherein performing a lookup for each of the pixels of the similarity transformed candidate patch comprises calculating pixel coordinates of each pixel in a patch row using one fused multiple-add vector instruction.

12. The non-transitory media of claim 11 , wherein performing the one or more patch validity tests includes executing the third patch validity test based on a determination that patch validity cannot be determined using either of the first or second patch validity tests.

13. The non-transitory media of claim 11 , wherein performing the one or more patch validity tests includes executing a fringe test for range invalidity comprising:

adding a fringe to the source image; and

verifying the similarity transformed candidate patch based on performing a lookup to determine whether one of the pixels of the similarity transformed candidate patch falls within the fringe.

14. The non-transitory media of claim 13 , wherein performing the one or more patch validity tests includes executing the fringe test for range invalidity as a precursor to each of the first, second and third patch validity tests.

15. The non-transitory media of claim 11 , automatically reconstructing the target image using the validated similarity transformed candidate patch.

16. A computer system comprising:

one or more hardware processors and memory configured to provide computer program instructions to the one or more hardware processors;

a patch-based synthesizer configured to generate a similarity transformed candidate patch for reconstructing a target image, wherein the similarity transformed candidate patch comprises pixels from a source image;

a means for verifying that the similarity transformed candidate patch falls within a designated sampling region of the source image by performing one or more patch validity tests comprising an evaluation of a representative pixel of the similarity transformed candidate patch; and

automatically reconstructing the target image using the validated similarity transformed candidate patch.

17. The computer system of claim 16 , wherein the means for verifying is configured to execute a comprehensive pixel test for patch validity based on a determination that patch validity cannot be determined using either of a hole dilation test for patch invalidity or a no-dilation test for patch invalidity.

18. The computer system of claim 17 , wherein the means for verifying is configured to execute a fringe test for range invalidity by:

adding a fringe to the source image; and

verifying the similarity transformed candidate patch based on performing a lookup to determine whether one of the pixels of the similarity transformed candidate patch falls within the fringe.

19. The computer system of claim 17 , wherein the means for verifying is configured to execute the fringe test for range invalidity as a precursor to each of the one or more patch validity tests.

20. The computer system of claim 16 , wherein the representative pixel is a center pixel of the candidate patch.

Assignments (2)
CHANGE OF NAME Recorded Nov 29, 2018
From: ADOBE SYSTEMS INCORPORATED
To: ADOBE INC.
Reel/Frame 047687/0115 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 15, 2018
From: AMIRGHODSI, SOHRAB; WAMPLER, KEVIN; SHECHTMAN, ELYA; DARABI, ALIAKBAR
To: ADOBE SYSTEMS INCORPORATED
Reel/Frame 045806/0256 →
Continuity (1)
Related Publication 20190287225A1 · Sep 19, 2019