IP Library › Granted Patent US 9,076,221
Granted Patent B2
US 9,076,221 · App. 13/945,609 · Granted Jul 7, 2015

Removing an object from an image

Inventors: Yingen Xiong (Mountain View, CA); Tao Zhang (Sunnyvale, CA); Ning Bi (San Diego, CA)
Assignee: QUALCOMM Incorporated
G06T7/0081G06F3/0482G06F3/0488G06K9/4609G06K9/4652G06T5/005H04N1/3872G06T2207/20144
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,076,221
App. No.
13/945,609
Granted
Jul 7, 2015
Kind
B2
Abstract

A method for removing an object from an image is described. The image is separated into a source region and a target region. The target region includes the object to be removed. A contour of the target region may be extracted. One or more filling candidate pixels are obtained. Multiple filling patches are obtained. Each filling patch is centered at a filling candidate pixel. A filling patch may be selected for replacement.

Claims (71)

1. A method for removing an object from an image, comprising:

separating, by an electronic device, the image into a source region and a target region comprising the object to be removed;

determining a set of pixels corresponding to a contour of the target region, wherein the contour is recalculated at each iteration;

obtaining, by the electronic device, one or more filling candidate pixels;

obtaining, by the electronic device, multiple filling patches, wherein each filling patch is centered at a filling candidate pixel; and

selecting, by the electronic device, a filling patch for replacement.

2. The method of claim 1 , wherein the selected filling patch comprises a largest number of pixels in a filling patch source region of the selected filling patch.

3. The method of claim 1 , further comprising:

determining a replacement patch that is most similar to the selected filling patch; and

replacing the target region portion of the selected filling patch with the corresponding part of the replacement patch.

4. The method of claim 3 , wherein the replacement patch is selected to maximize correlation of both color information and texture information between the replacement patch and the selected filling patch.

5. The method of claim 3 , wherein the replacement patch is selected to maximize correlation between both grayscale information and gradient information between the replacement patch and the selected filling patch.

6. The method of claim 3 , further comprising iterating the method if all target region pixels are not filled.

7. The method of claim 3 , wherein the replacement patch is determined using a fast best patch matching algorithm.

8. The method of claim 3 , wherein the replacement patch is determined using one or more components that are weighted, wherein the components comprise color information, grayscale information, texture information and gradient information.

9. The method of claim 8 , wherein the components are weighted corresponding to an edgy image, a smooth image and a normal image.

10. The method of claim 1 , further comprising adjusting a resolution at one or more iterations of the method.

11. The method of claim 1 , wherein the source region and the target region are separated based on a touchscreen input.

12. The method of claim 1 , wherein the contour of the target region comprises a decimated set of pixels corresponding to the contour.

13. The method of claim 1 , wherein the contour of the target region comprises an upscaled set of pixels corresponding to the contour.

14. The method of claim 1 , wherein the target region is selected using a user interface that comprises a hidden pull-down menu.

15. An electronic device, comprising:

circuitry for separating an image into a source region and a target region comprising an object to be removed;

circuitry for determining a set of pixels corresponding to a contour of the target region, wherein the contour is recalculated at each iteration;

circuitry for obtaining one or more filling candidate pixels;

circuitry for obtaining multiple filling patches, wherein each filling patch is centered at a filling candidate pixel; and

circuitry for selecting a filling patch for replacement.

16. The electronic device of claim 15 , wherein the selected filling patch comprises a largest number of pixels in a filling patch source region of the selected filling patch.

17. The electronic device of claim 15 , further comprising:

circuitry for determining a replacement patch that is most similar to the selected filling patch; and

circuitry for replacing the target region portion of the selected filling patch with the corresponding part of the replacement patch.

18. The electronic device of claim 17 , wherein the replacement patch is selected to maximize correlation of both color information and texture information between the replacement patch and the selected filling patch.

19. The electronic device of claim 17 , wherein the replacement patch is selected to maximize correlation between both grayscale information and gradient information between the replacement patch and the selected filling patch.

20. The electronic device of claim 17 , wherein the replacement patch is determined using a fast best patch matching algorithm.

21. The electronic device of claim 17 , wherein the replacement patch is determined using one or more components that are weighted, wherein the components comprise color information, grayscale information, texture information and gradient information.

22. The electronic device of claim 21 , wherein the components are weighted corresponding to an edgy image, a smooth image and a normal image.

23. The electronic device of claim 15 , further comprising circuitry configured for adjusting a resolution at one or more iterations.

24. The electronic device of claim 15 , wherein the source region and the target region are separated based on a touchscreen input.

25. The electronic device of claim 15 , wherein the contour of the target region comprises a decimated set of pixels corresponding to the contour.

26. The electronic device of claim 15 , wherein the contour of the target region comprises an upscaled set of pixels corresponding to the contour.

27. The electronic device of claim 15 , wherein the target region is selected using a user interface that comprises a hidden pull-down menu.

28. An apparatus, comprising:

means for separating an image into a source region and a target region comprising an object to be removed;

means for determining a set of pixels corresponding to a contour of the target region, wherein the contour is recalculated at each iteration;

means for obtaining one or more filling candidate pixels;

means for obtaining multiple filling patches, wherein each filling patch is centered at a filling candidate pixel; and

means for selecting a filling patch for replacement.

29. The apparatus of claim 28 , wherein the selected filling patch comprises a largest number of pixels in a filling patch source region of the selected filling patch.

30. The apparatus of claim 28 , further comprising:

means for determining a replacement patch that is most similar to the selected filling patch; and

means for replacing the target region portion of the selected filling patch with the corresponding part of the replacement patch.

31. The apparatus of claim 30 , wherein the replacement patch is selected to maximize correlation of both color information and texture information between the replacement patch and the selected filling patch.

32. The apparatus of claim 30 , wherein the replacement patch is selected to maximize correlation between both grayscale information and gradient information between the replacement patch and the selected filling patch.

33. The apparatus of claim 30 , wherein the replacement patch is determined using a fast best patch matching algorithm.

34. The apparatus of claim 30 , wherein the replacement patch is determined using one or more components that are weighted, wherein the components comprise color information, grayscale information, texture information and gradient information.

35. The apparatus of claim 34 , wherein the components are weighted corresponding to an edgy image, a smooth image and a normal image.

36. The apparatus of claim 28 , further comprising adjusting a resolution at one or more iterations.

37. The apparatus of claim 28 , wherein the source region and the target region are separated based on a touchscreen input.

38. The apparatus of claim 28 , wherein the contour of the target region comprises a decimated set of pixels corresponding to the contour.

39. The apparatus of claim 28 , wherein the contour of the target region comprises an upscaled set of pixels corresponding to the contour.

40. The apparatus of claim 28 , wherein the target region is selected using a user interface that comprises a hidden pull-down menu.

41. A computer-program product comprising a non-transitory tangible computer-readable medium having instructions thereon, the instructions comprising:

code for causing an electronic device to separate an image into a source region and a target region comprising an object to be removed;

code for determining a set of pixels corresponding to a contour of the target region, wherein the contour is recalculated at each iteration;

code for causing the electronic device to obtain one or more filling candidate pixels;

code for causing the electronic device to obtain multiple filling patches, wherein each filling patch is centered at a filling candidate pixel; and

code for causing the electronic device to select a filling patch for replacement.

42. The computer-program product of claim 41 , wherein the selected filling patch comprises a largest number of pixels in a filling patch source region of the selected filling patch.

43. The computer-program product of claim 42 , further comprising:

code for causing the electronic device to determine a replacement patch that is most similar to the selected filling patch; and

code for causing the electronic device to replace the target region portion of the selected filling patch with the corresponding part of the replacement patch.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 27, 2013
From: XIONG, YINGEN; ZHANG, TAO; BI, NING
To: QUALCOMM INCORPORATED
Reel/Frame 031090/0952 →
Continuity (2)
Provisional Application 61739666 · Dec 19, 2012
Related Publication 20140169667A1 · Jun 19, 2014