IP Library Granted Patent US 12,014,511
Granted Patent B2
US 12,014,511 · App. 17/634,491 · Granted Jun 18, 2024

Sample imaging via two-pass light-field reconstruction

Inventors: Raimund Leitner (Puch/Hallein, AT); Jan Steinbrener (Puch/Hallein, AT)
Assignee: Molecular Devices (Austria) GmbH
G06T7/557G02B21/361G02B21/365G06T3/10G06T2207/10012G06T2207/10052
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,014,511
App. No.
17/634,491
Granted
Jun 18, 2024
Kind
B2
Abstract

Methods and systems for sample imaging via two-pass light-field reconstruction. In an exemplary method, a light-field image of a sample may be captured in a light-field plane. The light-field image may be forward-projected computationally to each of a plurality of z-planes in object space to generate a set of forward-projected z-plane images. Backward-projections computationally to the light-field plane of the same xy-region in object space from each z-plane image may be compared with the light-field image, to determine a respective degree of correspondence between the backward-projected xy-region from each of the z-plane images and the light-field image. For each different xy-region, at least one of the forward-projected z-plane images may be selected to contribute data for the different xy-region in a 2D or 3D object-space image of the sample.

Claims (33)

1. A method of imaging a sample, the method comprising:

(a) capturing a light-field image of the sample in a light-field plane;

(b) forward-projecting the light-field image computationally to each of a plurality of z-planes in object space to generate a set of z-plane images;

(c) comparing, with respect to the light-field image, a backward projection to the light-field plane of the same xy-region in object space from each z-plane image, to determine a respective degree of correspondence between the backward-projected xy-region from each of the z-plane images and the light-field image, wherein comparing is repeated for each of a plurality of different xy-regions in object space;

(d) selecting based on comparing, for each different xy-region, at least one of the z-plane images to contribute data for the different xy-region in an object-space image of the sample; and

(e) generating the object-space image of the sample.

2. The method of claim 1 , wherein each of the different xy-regions corresponds to a single pixel in each of the z-plane images of the set.

3. The method of claim 1 , wherein at least one of the different xy-regions corresponds to a group of pixels or an object.

4. The method of claim 1 , wherein the object-space image generated is a two-dimensional image.

5. The method of claim 4 , wherein the two-dimensional image is substantially all in focus.

6. The method of claim 1 , wherein the object-space image is a three-dimensional image.

7. The method of claim 6 , wherein the three-dimensional image is substantially all in focus.

8. The method of claim 1 , wherein selecting includes selecting, for at least one of the different xy-regions, only one of the z-plane images for which the degree of correspondence is higher than for each other z-plane image of the set.

9. The method of claim 1 , wherein selecting includes selecting, for at least one of the different xy-regions, at least two of the z-plane images for which the corresponding degrees of correspondence of backward projection are higher than for each other z-plane image of the set.

10. The method of claim 9 , further comprising comparing the respective degrees of correspondence for backward projection of a given xy-region to a threshold, wherein selecting includes selecting each z-plane image for which comparing the respective degrees of correspondence meets a predefined condition.

11. The method of claim 1 , wherein selecting includes selecting a best z-plane in object space for groups of pixels or single pixels in object space, and wherein generating includes generating a substantially all-in-focus three-dimensional image of the sample.

12. The method of claim 1 , wherein selecting includes selecting a best z-plane in object space for groups of pixels or single pixels in object space, and wherein generating includes generating a substantially all-in-focus two-dimensional image of the sample.

13. The method of claim 1 , wherein selecting includes selecting two or more z-planes in object space for each of a plurality of single pixels or groups of pixels in object space, further comprising combining information from the two or more z-planes selected for each single pixel or group of pixels.

14. The method of claim 13 , wherein combining information includes averaging or interpolating values for the single pixel or group of pixels from the two or more z-planes selected.

15. The method of claim 13 , wherein combining information is based on the respective degrees of correspondence for the single pixel or group of pixels from the two or more z-planes selected.

16. The method of claim 1 , further comprising computationally backward-projecting each of the z-plane images of the set to the light-field plane to obtain a set of backward-projected images, wherein comparing uses the set of backward-projected images.

17. The method of claim 1 , wherein capturing is performed by an image sensor of an imaging system, and wherein forward-projecting and each backward projection uses a known mapping geometry for the imaging system.

18. The method of claim 17 , wherein the known mapping geometry utilizes a value for a total magnification of the imaging system and at least one value for a magnification produced by a microlens array of the imaging system.

19. A system for imaging a sample, comprising:

(a) a stage to hold a sample on an optical axis in object space,

(b) a microlens array on the optical axis;

(c) an image sensor configured to capture a light-field image of the sample formed with light that has propagated through the microlens array; and

(d) a processor configured to

(i) forward-project the light-field image computationally to each of a plurality of z-planes in object space to generate a set of z-plane images,

(ii) compare, with respect to the light-field image, a backward projection to the light-field plane of the same xy-region in object space from each z-plane image, to determine a respective degree of correspondence between the backward-projected xy-region from each of the z-plane images and the light-field image, wherein comparing is repeated for each of a plurality of different xy-regions in object space,

(iii) select for each different xy-region at least one of the z-plane images to contribute data for the different xy-region in an object-space image of the sample, and

(iv) generate the object-space image of the sample.

20. A non-transitory computer readable medium having instructions which cause a processor to perform the method of claim 1 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 28, 2022
From: LEITNER, RAIMUND; STEINBRENER, JAN
To: MOLECULAR DEVICES (AUSTRIA) GMBH
Reel/Frame 059113/0726 →
Continuity (2)
Provisional Application 62885735 · Aug 12, 2019
Related Publication 20220270279A1 · Aug 25, 2022
Cited By (1)
US 12,598,400