IP Library Granted Patent US 9,124,797
Granted Patent B2
US 9,124,797 · App. 13/170,192 · Granted Sep 1, 2015

Image enhancement via lens simulation

Inventors: Brian K. Guenter (Redmond, WA); Neel S. Joshi (Seattle, WA)
Assignee: Microsoft Technology Licensing, LLC
H04N5/23212H04N5/3572
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,124,797
App. No.
13/170,192
Filed
Jun 28, 2011
Granted
Sep 1, 2015
Kind
B2
Art Unit
2664
USPC
348/345
Abstract

The described implementations relate to enhancing images. One system includes a lens configured to focus an image on an image sensor. The image sensor is configured to capture the image. The model of the lens can be known to the system. The system can also include an image enhancement component configured to receive the captured image and to utilize a simulated blur kernel of the known model to produce an enhanced image.

Claims (37)

1. A system, comprising:

an individual lens of a known model, wherein the individual lens is configured to focus an image on an image sensor to capture the image and the individual lens shares lens model parameters with multiple other lenses of the known model;

a hardware processor; and

hardware storage storing instructions which, when executed by the hardware processor, cause the hardware processor to:

obtain the captured image;

acquire a simulated blur kernel for the known model of the individual lens, wherein the simulated blur kernel addresses lens model aberrations associated with the lens model parameters shared with the multiple other lenses of the known model, the simulated blur kernel having been produced using a numerical approximation of a diffraction integral computed by summing complex phasors of pixels in an image plane sampling window; and

utilize the simulated blur kernel of the known model to produce an enhanced image from the captured image that accounts for the lens model aberrations.

2. The system of claim 1 , wherein the instructions cause the hardware processor to:

perform the numerical approximation to acquire the simulated blur kernel.

3. The system of claim 2 , wherein the numerical approximation of the diffraction integral used to produce the simulated blur kernel accounts for glass formulas used by the individual lens and the multiple other lenses of the known model.

4. The system of claim 2 , wherein the numerical approximation of the diffraction integral used to produce the simulated blur kernel accounts for a radius of curvature shared by the individual lens and the multiple other lenses of the known model.

5. The system of claim 2 , wherein the numerical approximation of the diffraction integral used to produce the simulated blur kernel accounts for lens element diameter shared by the individual lens and the multiple other lenses of the known model.

6. The system of claim 2 , wherein the numerical approximation of the diffraction integral used to produce the simulated blur kernel accounts for an antireflection coating material and thickness shared by the individual lens and the multiple other lenses of the known model.

7. A system comprising:

a hardware processor; and

hardware storage storing instructions which, when executed by the hardware processor, cause the hardware processor to:

obtain an image captured through an individual lens, wherein the individual lens is one of several different lenses of a known lens model;

access a point spread function derived from a wave optic simulation of the known lens model, wherein the wave optic simulation uses a numerical approximation that addresses shared characteristics associated with the several different lenses of the lens model to compute the point spread function; and,

use a probabilistic technique to find a likely estimate of a sharp image given the captured image and a blur matrix derived using the point spread function.

8. The system of claim 7 , embodied as an imaging device comprising the individual lens.

9. The system of claim 7 , wherein the instructions further cause the hardware processor to:

perform the wave optic simulation and the numerical approximation, wherein the numerical approximation approximates a diffraction integral.

10. The system of claim 9 , wherein the numerical approximation is calculated using a wavelength or wavenumber parameter of the diffraction integral.

11. The system of claim 9 , wherein the numerical approximation of the diffraction integral is calculated using a field intensity parameter of the diffraction integral, the field intensity parameter reflecting a field intensity at a point on a wavefront.

12. The system of claim 9 , wherein the numerical approximation of the diffraction integral is calculated using a distance parameter of the diffraction integral reflecting a distance between a point on a wavefront and a point in an image plane.

13. The system of claim 12 , wherein the numerical approximation of the diffraction integral is calculated using an angle parameter of the diffraction integral reflecting an angle between a normal to the wavefront and a vector defined by the point on a wavefront and the point in the image plane.

14. The system of claim 7 , wherein the captured image includes metadata that identifies an imaging device that captured the image and the instructions further cause the hardware processor to:

perform a refocusing process utilizing simulated defocus kernels obtained from the wave optic simulation to refocus the image.

15. The system of claim 14 , wherein the image is a shallow depth of field image and the refocusing process involves deblurring the shallow depth of field image to obtain a sharp image.

16. The system of claim 15 , wherein the refocusing process involves using the wave optics simulation to obtain the simulated defocus kernels for several different distances in front of and behind a focal plane of the shallow depth of field image.

17. The system of claim 16 , wherein the refocusing process involves finding a labeling for points in the image using kurosis of the image in a pixel window as a data cost and minimizing or reducing the data cost subject to a smoothness constraint.

18. A method performed using a hardware processor, the method comprising:

obtaining an image captured with an individual lens of a known model, wherein the individual lens shares lens model parameters with multiple other lenses of the known model;

acquiring a simulated blur kernel for the known model of the individual lens, wherein the simulated blur kernel addresses lens model aberrations associated with the lens model parameters that are shared with the multiple other lenses of the known model, the simulated blur kernel having been produced using a numerical approximation of a diffraction integral computed using sums of complex phasors of pixels in an image plane sampling window; and

utilizing the simulated blur kernel of the known model to produce an enhanced version of the image that accounts for the lens model aberrations.

19. The method of claim 18 , further comprising selecting the simulated blur kernel from a matrix of multiple sampled blur kernels produced using the numerical approximation.

20. The method of claim 18 , wherein the utilizing comprises performing a deconvolution operation to produce the enhanced version of the image.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034544/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 28, 2011
From: GUENTER, BRIAN K.; JOSHI, NEEL S.
To: MICROSOFT CORPORATION
Reel/Frame 026510/0373 →
Continuity (1)
Related Publication 20130002932A1 · Jan 3, 2013