IP Library Granted Patent US 10,032,254
Granted Patent B2
US 10,032,254 · App. 13/825,365 · Granted Jul 24, 2018

Method and device for recovering a digital image from a sequence of observed digital images

Inventors: Stefan Harmeling (Tübingen, DE); Michael Hirsch (Tübingen, DE); Suvrit Sra (Tübingen, DE); Bernhard Schölkopf (Tübingen, DE); Christian J. Schuler (Tübingen, DE)
Assignee: Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V.
G06T5/001G06T5/003H04N5/23229G06T2207/20201H04N5/23267H04N5/3572
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,032,254
App. No.
13/825,365
Granted
Jul 24, 2018
Kind
B2
Abstract

A computer-implemented method for recovering a digital image (x) from a sequence of observed digital images (y 1 , . . . , y T ), includes: obtaining an observed digital image (y t ); estimating a point spread function (f t ) based on the observed image (y t ); estimating the recovered digital image (x), based on the estimated point spread function (f t ) and the observed image (y t ); and repeating the above steps. In order to correct optical aberrations of a lens, a point spread function of the lens may be used.

Claims (51)

1. A computer-implemented method for recovering a digital image from an observed digital image acquired by an image acquisition device, the computer-implemented method comprising:

acquiring the observed image;

dividing the observed digital image into at least a first patch and a second patch, wherein the first patch overlaps with the second patch;

for the first patch,

applying a first windowing function to the first patch to obtain a first windowed patch; and

processing the first windowed patch with a first filter having a first point spread function to obtain a first transformed patch;

for the second patch,

applying a second windowing function to the second patch to obtain a second windowed patch; and

processing the second windowed patch with a second filter having a second point spread function to obtain a second transformed patch;

estimating a recovered digital image from at least the first transformed patch and the second transformed patch of the observed digital image; and

outputting the recovered digital image.

2. The computer-implemented method according to claim 1 , wherein the recovered digital image has a higher resolution than the observed digital image.

3. The computer-implemented method according to claim 1 , wherein the observed digital image includes a saturated pixel, and wherein the saturated pixel is disregarded in processing the first windowed patch with the first filter.

4. The computer-implemented method according to claim 1 , wherein the observed image includes an individual color channel, and wherein at least the first point spread function models the individual color channel.

5. The computer-implemented method according to claim 1 , wherein the observed image includes multiple color channels, and wherein at least the first point spread function models the multiple color channels.

6. The computer-implemented method according to claim 1 , further comprising removing noise from the observed image, wherein removing noise from the observed image comprises:

applying a bilateral filter to at least a portion of the observed image to preserve an edge of the portion; and

applying a shock filter to the at least a portion of the observed digital image to emphasize the edge of the portion.

7. The computer-implemented method according to claim 1 , wherein:

dividing the observed digital image further comprises dividing the observed digital image into at least the first patch, the second patch and a third patch,

the computer-implemented further comprises:

for the third patch,

applying a third windowing function to the third patch to obtain a third windowed patch; and

processing the third windowed patch with a third filter having a third point spread function to obtain a third transformed patch; and

estimating the recovered digital image further comprises estimating the recovered digital image from at least the first transformed patch, the second transformed patch and the third transformed patch of the observed digital image.

8. The computer-implemented method according to claim 1 , wherein the observed digital image is an astronomical image.

9. The computer-implemented method according to claim 1 , wherein the observed digital image is magnetic resonance image (MRI).

10. The computer-implemented method according to claim 2 , wherein the first filter is a space-variant filter.

11. A computing device for recovering a digital image from an observed digital image acquired by an image acquisition device, the computing device comprising:

a memory configured to at least store specific computer-executable instructions; and

a processor in communication with the memory, wherein the processor is configured to execute the specific computer-executable instructions to at least:

divide the observed digital image into a plurality of patches, wherein in at least one patch of the plurality of patches overlaps with another patch of the plurality of patches;

for each patch of the plurality of patches,

apply a windowing function to the patch to obtain a windowed patch; and

process the windowed patch with a filter based at least partly on a point spread function to obtain a transformed patch, wherein the filter with which a first windowed patch of the plurality of patches is processed, is different from the filter with which a second windowed patch of the plurality of patches is processed; and

estimate a recovered digital image from the transformed patches of the observed digital image; and

output the recovered digital image.

12. A non-transitory computer-readable medium having a computer-executable module configured to execute in one or more processors, the computer-executable module being further configured to at least:

divide an observed digital image acquired by an image acquisition device into-at least a first patch and a second patch, wherein the first patch overlaps with the second patch;

for the first patch,

apply a first windowing function to the first patch to obtain a first windowed patch; and

process the first windowed patch with a first filter having a first point spread function to obtain a first transformed patch;

for the second patch,

apply a second windowing function to the second patch to obtain a second windowed patch; and

process the second windowed patch with a second filter having a second point spread function to obtain a second transformed patch;

estimate a recovered digital image from at least the first transformed patch and the second transformed patch of the observed digital image; and

output the recovered digital image.

13. The computer device according to claim 11 , wherein for each of the plurality of patches, the processor is configured to further execute the specific computer-executable instructions to at least estimate the point spread function, upon which the filter is at least partly based, from the observed image.

14. The computer device according to claim 11 , wherein for each of the plurality of patches, the processor is configured to further execute the specific computer-executable instructions to at least jointly estimate a plurality of point spread functions.

15. The computer device according to claim 14 , wherein the filter with which the windowed patch is processed, is based at least partly on the plurality of point spread functions.

16. The computer device according to claim 11 , wherein processing the windowed patch with the filter provides for a non-stationary convolution of the windowed patch.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 3, 2013
From: HARMELING, STEFAN; HIRSCH, MICHAEL; SRA, SUVRIT; SCHALKOPF, BERNHARD; SCHULER, CHRISTIAN J
To: MAX-PLANCK-GESELLSCHAFT ZUR FORDERUNG DER WISSENSCHAFTEN E.V.
Reel/Frame 030531/0342 →
Priority Claims (3)
EP 10181003 · Sep 28, 2010 · regional
EP 10194542 · Dec 10, 2010 · regional
EP 11176703 · Aug 5, 2011 · regional
Continuity (3)
Provisional Application 61387025 · Sep 28, 2010
Provisional Application 61421852 · Dec 10, 2010
Related Publication 20130242129A1 · Sep 19, 2013
Cited By (4)
US 12,216,277 US 12,393,765 US 12,482,075 US 12,651,320