IP Library Granted Patent US 7,689,036
Granted Patent B2
US 7,689,036 · App. 11/236,805 · Granted Mar 30, 2010

Correction of blotches in component images

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,689,036
App. No.
11/236,805
Granted
Mar 30, 2010
Kind
B2
Abstract

Blotches may be identified and processed to reduce or eliminate the blotch. The blotch may be in just one of several separations and multiple separations may be used, for example, to identify the blotch. An implementation (i) compares a first component image of an image with a first component image of a reference image, (ii) compares a second component image of the image with a second component image of the reference image, and (iii) determines based on these comparisons whether the first component image of the image includes a blotch. Multiple image separations also, or alternatively, may be used, for example, to modify the blotch, as well as to evaluate whether a modification is beneficial.

Claims (232)

1. A method of processing images containing a blotch, the method comprising:

using at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result; and

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result,

wherein comparing the first component image of the target image with the first component image of the reference image comprises determining a first-component difference between intensities of (i) a given pixel having a given location in the first component image of the target image and (ii) a corresponding pixel in the first component image of the reference image, the corresponding pixel in the first component image of the reference image being determined based on motion indicated by the motion vector,

wherein comparing the second component image of the target image with the second component image of the reference image comprises determining a second-component difference between intensities of (i) a pixel having the given location in the second component image of the target image and (ii) a corresponding pixel in the second component image of the reference image, the corresponding pixel in the second component image of the reference image being determined based on motion indicated by the motion vector, and

wherein determining that the first component image of the target image includes the blotch comprises identifying the given pixel as being part of the blotch based on the first-component difference and the second-component difference.

2. The method of claim 1 wherein the first component image of the target image comprises a color image.

3. The method of claim 2 wherein the second component image of the target image comprises a color image.

4. The method of claim 1 wherein the target image comprises a color image.

5. The method of claim 1 :

further comprising determining a third-component difference between intensities of (i) a pixel having the given location in a third component image of the target image and (ii) a corresponding pixel in a third component image of the reference image, the corresponding pixel in the third component image of the reference image being determined based on motion indicated by the motion vector, and

wherein identifying the given pixel as being part of the blotch comprises identifying the given pixel as being part of the blotch if:

(i) the first-component difference minus the second-component difference exceeds a first/second-component threshold,

(ii) the first-component difference minus the third-component difference exceeds a first/third-component threshold,

(iii) the second-component difference is less than a second-component threshold, and

(iv) the third-component difference is less than a third-component threshold.

6. The method of claim 1 wherein the determined motion vector is based on at least a portion of the first component image of the target image and at least a portion of the second component image of the target image.

7. A method of processing images containing a blotch, the method comprising:

using at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result; and

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result,

wherein determining the motion vector comprises:

determining a first value for first-component distortion characterizing distortion in the first component image of the target image associated with applying a first candidate motion vector to the first component image of the target image;

determining a first value for second-component distortion characterizing distortion in the second component image of the target image associated with applying the first candidate motion vector to the second component image of the target image;

determining a first value for a distortion function characterizing distortion from multiple component images of the target image associated with applying the first candidate motion vector to the multiple component images of the target image;

determining a second value for first-component distortion characterizing distortion in the second component image of the target image associated with applying a second candidate motion vector to the first component image of the target image;

determining a second value for second-component distortion characterizing distortion in the second component image of the target image associated with applying the second candidate motion vector to the second component image of the target image;

determining a second value for the distortion function characterizing distortion from multiple component images of the target image associated with applying the second candidate motion vector to the multiple component images of the target image; and

evaluating the second candidate motion vector based on the relative values of the first and second values of (1) first-component distortion, (2) second-component distortion, and (3) the distortion function.

8. The method of claim 7 wherein evaluating the second candidate motion vector comprises:

making a determination that the first value of at least one of (1) first-component distortion, (2) second-component distortion, and (3) the distortion function is less than the respective second value; and

favoring the second candidate motion vector based on the determination.

9. The method of claim 8 wherein making the determination further comprises making a determination that the first value of at least two of (1) first-component distortion, (2) second-component distortion, and (3) the distortion function is less than the respective second value.

10. The method of claim 7 wherein

determining a first value for third-component distortion characterizing distortion in a third component image of the target image associated with applying the first candidate motion vector to the third component image of the target image;

determining a second value for third-component distortion characterizing distortion in the third component image of the target image associated with applying the second candidate motion vector to the third component image of the target image;

making a component-distortion determination that the first value of at least two of (1) first-component distortion, (2) second-component distortion, and (3) third-component distortion are less than the respective second value;

making a distortion-function determination that the first value of the distortion function is less than the second value of the distortion function; and

favoring the second candidate motion vector based on the component-distortion determination and the distortion-function determination.

11. The method of claim 1 further comprising determining an extent of the blotch in the first component image of the target image.

12. The method of claim 11 further comprising refining the motion vector based on the determined extent of the blotch.

13. The method of claim 1 further comprising testing the motion vector using the second component image of the target image.

14. A method of processing images containing a blotch, the method comprising:

using at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result;

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result; and

testing the motion vector using the second component image of the target image,

wherein testing the motion vector using the second component image of the target image comprises:

determining an extent of the blotch in the first component image of the target image;

applying the motion vector to pixels in a corresponding extent of a digital representation of the second component image of the target image, the corresponding extent including pixels that correspond to the determined extent of the blotch;

determining a blotch distortion value for the application of the motion vector to the pixels in the corresponding extent;

determining a region in the first component image of the target image that includes more than the determined extent of the blotch;

applying the motion vector to pixels in the digital representation of the second component image of the target image that are in a corresponding region but not in the corresponding extent, the corresponding region includes pixels that correspond to the determined region;

determining a non-blotch distortion value for the application of the motion vector to pixels that are in the corresponding region but not in the corresponding extent; and

evaluating the motion vector based on the blotch distortion value and the non-blotch distortion value.

15. The method of claim 14 wherein evaluating the motion vector comprises:

determining a difference between the blotch distortion value and the non-blotch distortion value;

comparing the difference to a difference threshold; and

evaluating the motion vector based on a result obtained from comparing the difference to the difference threshold.

16. The method of claim 1 further comprising performing a spatial correction in the first component image of the target image to modify the blotch.

17. The method of claim 16 further comprising:

applying the spatial correction to the second component image of the target image;

determining a spatial-correction distortion value for the application of the spatial correction to the second component image of the target image;

comparing the spatial-correction distortion value to a spatial-correction distortion threshold; and

evaluating the spatial correction of the first component image of the target image based on a result obtained from comparing the spatial-correction distortion value to the spatial-correction distortion threshold.

18. A method of processing images containing a blotch, the method comprising:

using at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result;

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result;

performing a spatial correction in the first component image of the target image to modify the blotch;

applying the spatial correction to the second component image of the target image;

determining a spatial-correction distortion value for the application of the spatial correction to the second component image of the target image;

comparing the spatial-correction distortion value to a spatial-correction distortion threshold; and

evaluating the spatial correction of the first component image of the target image based on a result obtained from comparing the spatial-correction distortion value to the spatial-correction distortion threshold,

wherein determining the spatial-correction distortion value comprises determining a distortion value between (1) the second component image of the target image before applying the spatial correction and (2) the second component image of the target image after applying the spatial correction.

19. The method of claim 1 further comprising performing a correction in the first component image of the target image to modify the blotch.

20. The method of claim 19 further comprising evaluating the correction using the second component image of the target image.

21. A method of processing images containing a blotch, the method comprising:

using at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result;

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result;

performing a correction in the first component image of the target image to modify the blotch; and

evaluating the correction using the second component image of the target image,

wherein evaluating the correction using the second component image of the target image comprises:

determining a first-component distortion value characterizing distortion from performing the correction in the first component image of the target image;

performing the correction in the second component image of the target image;

determining a second-component distortion value characterizing distortion from performing the correction in the second component image of the target image;

comparing the first-component distortion value with the second-Component distortion value to produce a result; and

evaluating the correction in the first component image of the target image based on the result.

22. The method of claim 21 wherein the first-component distortion value is based on a ratio of (i) a variance, before modification, of a particular region in the first component image of the target image, the particular region including the blotch, and (ii) a variance of the particular region after modification of the blotch.

23. A method of processing images containing a blotch, the method comprising:

using at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result;

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result;

determining an extent of the blotch in the first component image of the target image;

determining a corresponding extent in the first component image of the reference image that corresponds to the extent of the blotch in the first component image of the target image;

modifying the blotch in the first component image of the target image using information from the corresponding extent in the first component image of the reference image;

determining a region in the first component image of the target image that includes more than the determined extent of the blotch;

determining a blotch distortion value for the determined region that includes the modification of the blotch in the first component image of the target image;

determining a corresponding extent in the second component image of the target image that corresponds to the extent in the first component image of the target image;

determining a corresponding extent in the second component image of the reference image that corresponds to the extent in the first component image of the target image;

modifying the corresponding extent in the second component image of the target image using information from the corresponding extent in the second component image of the reference image;

determining a corresponding region in the second component image of the target image that corresponds to the determined region;

determining a corresponding-region distortion value for the corresponding region that includes the modification of the corresponding extent in the second component image of the target image; and

evaluating the modification of the blotch in the first component image of the target image based on the blotch distortion value and the corresponding-region distortion value.

24. An apparatus for processing images containing a blotch, the apparatus comprising:

at least one processor; and

at least one computer-readable medium coupled to the at least one processor having instructions stored thereon which, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result; and

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result,

wherein comparing the first component image of the target image with the first component image of the reference image comprises determining a first-component difference between intensities of (i) a given pixel having a given location in the first component image of the target image and (ii) a corresponding pixel in the first component image of the reference image, the corresponding pixel in the first component image of the reference image being determined based on motion indicated by the motion vector,

wherein comparing the second component image of the target image with the second component image of the reference image comprises determining a second-component difference between intensities of (i) a pixel having the given location in the second component image of the target image and (ii) a corresponding pixel in the second component image of the reference image, the corresponding pixel in the second component image of the reference image being determined based on motion indicated by the motion vector, and

wherein determining that the first component image of the target image includes the blotch comprises identifying the given pixel as being part of the blotch based on the first component difference and the second-component difference.

25. An apparatus for processing images containing a blotch, the apparatus comprising:

at least one processor: and

at least one computer-readable medium coupled to the at least one processor having instructions stored thereon which, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result; and

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result,

wherein determining the motion vector comprises:

determining a first value for first-component distortion characterizing distortion in the first component image of the target image associated with applying a first candidate motion vector to the first component image of the target image;

determining a first value for second-component distortion characterizing distortion in the second component image of the target image associated with applying the first candidate motion vector to the second component image of the target image;

determining a first value for a distortion function characterizing distortion from multiple component images of the target image associated with applying the first candidate motion vector to the multiple component images of the target image;

determining a second value for first-component distortion characterizing distortion in the second component image of the target image associated with applying a second candidate motion vector to the first component image of the target image;

determining a second value for second-component distortion characterizing distortion in the second component image of the target image associated with applying the second candidate motion vector to the second component image of the target image;

determining a second value for the distortion function characterizing distortion from multiple component images of the target image associated with applying the second candidate motion vector to the multiple component images of the target image; and

evaluating the second candidate motion vector based on the relative values of the first and second values of (1) first-component distortion, (2) second-component distortion, and (3) the distortion function.

26. The apparatus of claim 25 wherein evaluating the second candidate motion vector comprises:

making a determination that the first value of at least one of (1) first-component distortion, (2) second-component distortion, and (3) the distortion function is less than the respective second value; and

favoring the second candidate motion vector based on the determination.

27. An apparatus for processing images containing a blotch, the apparatus comprising:

at least one processor; and

at least one computer-readable medium coupled to the at least one processor having instructions stored thereon which, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result;

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result; and

testing the motion vector using the second component image of the target image,

wherein testing the motion vector using the second component image of the target image comprises:

determining an extent of the blotch in the first component image of the target image;

applying the motion vector to pixels in a corresponding extent of a digital representation of the second component image of the target image, the corresponding extent including pixels that correspond to the determined extent of the blotch;

determining a blotch distortion value for the application of the motion vector to the pixels in the corresponding extent;

determining a region in the first component image of the target image that includes more than the determined extent of the blotch;

applying the motion vector to pixels in the digital representation of the second component image of the target image that are in a corresponding region but not in the corresponding extent, the corresponding region includes pixels that correspond to the determined region;

determining a non-blotch distortion value for the application of the motion vector to pixels that are in the corresponding region but not in the corresponding extent; and

evaluating the motion vector based on the blotch distortion value and the non blotch distortion value.

28. An apparatus for processing images containing a blotch, the apparatus comprising:

at least one processor; and

at least one computer-readable medium coupled to the at least one processor having instructions stored thereon which, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result;

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result;

performing a spatial correction in the first component image of the target image to modify the blotch;

applying the spatial correction to the second component image of the target image;

determining a spatial-correction distortion value for the application of the spatial correction to the second component image of the target image;

comparing the spatial-correction distortion value to a spatial-correction distortion threshold; and

evaluating the spatial correction of the first component image of the target image based on a result obtained from comparing the spatial-correction distortion value to the spatial-correction distortion threshold,

wherein determining the spatial-correction distortion value comprises determining a distortion value between (1) the second component image of the target image before applying the spatial correction and (2) the second component image of the target image after applying the spatial correction.

29. An apparatus for processing images containing a blotch, the apparatus comprising:

at least one processor; and

at least one computer-readable medium coupled to the at least one processor having instructions stored thereon which, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result;

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result;

performing a correction in the first component image of the target image to modify the blotch; and

evaluating the correction using the second component image of the target image,

wherein evaluating the correction using the second component image of the target image comprises:

determining a first-component distortion value characterizing distortion from performing the correction in the first component image of the target image;

performing the correction in the second component image of the target image;

determining a second-component distortion value characterizing distortion from performing the correction in the second component image of the target image;

comparing the first-component distortion value with the second-component distortion value to produce a result; and

evaluating the correction in the first component image of the target image based on the result.

30. An apparatus for processing images containing a blotch, the apparatus comprising:

at least one processor; and

at least one computer-readable medium coupled to the at least one processor having instructions stored thereon which, when executed by the at least one processor, causes the at least one processor to perform operations comprising:

accessing one or more component images of a target image, the target image including a first component image and a second component image;

accessing one or more component images of a reference image, the reference image including a first component image and a second component image;

determining a motion vector characterizing motion between the target image and the reference image;

comparing, based on motion indicated by the motion vector, the first component image of the target image with the first component image of the reference image to produce a first-component comparison result;

comparing, based on motion indicated by the motion vector, the second component image of the target image with the second component image of the reference image to produce a second-component comparison result;

determining that the first component image of the target image includes a blotch based on the first-component comparison result and the second-component comparison result;

determining an extent of the blotch in the first component image of the target image;

determining a corresponding extent in the first component image of the reference image that corresponds to the extent of the blotch in the first component image of the target image;

modifying the blotch in the first component image of the target image using information from the corresponding extent in the first component image of the reference image;

determining a region in the first component image of the target image that includes more than the determined extent of the blotch;

determining a blotch distortion value for the determined region that includes the modification of the blotch in the first component image of the target image;

determining a corresponding extent in the second component image of the target image that corresponds to the extent in the first component image of the target image;

determining a corresponding extent in the second component image of the reference image that corresponds to the extent in the first component image of the target image;

modifying the corresponding extent in the second component image of the target image using information from the corresponding extent in the second component image of the reference image;

determining a corresponding region in the second component image of the target image that corresponds to the determined region;

determining a corresponding-region distortion value for the corresponding region that includes the modification of the corresponding extent in the second component image of the target image; and

evaluating the modification of the blotch in the first component image of the target image based on the blotch distortion value and the corresponding-region distortion value.

Assignments (6)
SECURITY INTEREST Recorded Oct 1, 2025
From: WARNER BROS. DISCOVERY, INC.; WARNER MEDIA, LLC; TURNER BROADCASTING SYSTEM, INC.; HOME BOX OFFICE, INC.; DISCOVERY COMMUNICATIONS, LLC; WARNERMEDIA DIRECT LLC; DISCOVERY.COM LLC; WARNER BROS. ENTERTAINMENT INC.; CNN INTERACTIVE GROUP, INC.
To: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
Reel/Frame 072995/0858 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS Recorded Nov 16, 2010
From: BANK OF AMERICA, N A
To: AOL INC; AOL ADVERTISING INC; GOING INC; LIGHTNINGCAST LLC; MAPQUEST, INC; NETSCAPE COMMUNICATIONS CORPORATION; QUIGO TECHNOLOGIES LLC; SPHERE SOURCE, INC; TACODA LLC; TRUVEO, INC; YEDDA, INC
Reel/Frame 025323/0416 →
SECURITY AGREEMENT Recorded Dec 14, 2009
From: AOL INC.; AOL ADVERTISING INC.; BEBO, INC.; ICQ LLC; GOING, INC.; LIGHTNINGCAST LLC; MAPQUEST, INC.; NETSCAPE COMMUNICATIONS CORPORATION; QUIGO TECHNOLOGIES LLC; SPHERE SOURCE, INC.; TACODA LLC; TRUVEO, INC.; YEDDA, INC.
To: BANK OF AMERICAN, N.A. AS COLLATERAL AGENT
Reel/Frame 023649/0061 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 28, 2009
From: AMERICA ONLINE, INC.
To: AOL LLC
Reel/Frame 023435/0452 →
JOINT UNDIVIDED ASSIGNMENT AGREEMENT Recorded Oct 28, 2009
From: AOL LLC
To: WARNER BROS. ENTERTAINMENT INC.
Reel/Frame 023435/0582 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2006
From: PERLMUTTER, KEREN O.; PERLMUTTER, SHARON M.
To: AMERICA ONLINE, INC.
Reel/Frame 018013/0811 →