Image forming method and microscope device
More images than spatial modulation components of a lighting light are acquired, and a least square method is applied to the imaged plurality of images to obtain signal components. Accordingly, a ultra-resolution microscope which uses a spatially-modified lighting light to illuminate a sample, arithmetic-processes acquired images to thereby demodulate a given spatial modulation, and obtains a high-resolution sample image, wherein a high-resolution image reduced in noise components can be obtained.
1. An image forming method in which a sample is illuminated with sinusoidally spatially modulated illuminating light,
light from the sample illuminated by the illuminating light is focused as an image, and a sample image is formed by image calculation processing from the acquired image, wherein
a greater number of images than the number of spatially modulated components contained in the light emitted from the sample as a result of the illuminating light on the sample are acquired, and the sample image is formed by applying the method of least squares, which is shown by the following equation and in which complex numbers are considered,
(
∑
j
b
1
j
I
kj
(
k
)
∑
j
b
0
j
I
kj
(
k
)
∑
j
b
+
1
j
I
kj
(
k
)
)
=
(
∑
j
b
-
1
j
2
∑
j
b
-
1
j
b
0
j
∑
j
b
-
1
j
b
+
1
j
∑
j
b
0
j
b
-
1
j
∑
j
b
0
j
2
∑
j
b
0
j
b
+
1
j
∑
j
b
+
1
j
b
-
1
j
∑
j
b
+
1
j
b
0
j
∑
j
b
+
1
j
2
)
(
O
k
(
k
-
K
)
P
k
(
k
)
O
k
(
k
)
P
k
(
k
)
O
k
(
k
+
K
)
P
k
(
k
)
)
(
1
)
where
b lj =m l exp(iφ j ),
m l is a modulated amplitude of the illuminating light,
φ j is a modulated phase of the jth one of images,
I kj (k) is a signal intensity of the jth one of images, and
P k (k) is a transfer function of the optical system,
with the amount by which the plurality of picked-up images are varied when these images are picked up for each of the spatial frequency components constituting the plurality of picked-up images taken as a parameter.
2. The image forming method according to claim 1 , wherein the plurality of images are acquired while varying the phase of the illuminating light pattern on the sample.
3. The image forming method according to claim 1 , wherein the plurality of images are acquired while varying the modulation amplitude of the illuminating light on the sample.
4. The image forming method according to claim 1 , wherein spatially modulated illuminating lights are in three directions and one zero-order light modulated component, three +first-order light modulated components and three −first-order light modulated components are separated by applying the method of least squares, which is shown by the following equation in place of Equation (1)
(
∑
j
=
j
1
,
j
2
,
j
3
I
kj
(
k
)
∑
j
1
I
kj
1
(
k
)
exp
(
-
ⅈ
ϕ
j
1
)
∑
j
1
I
kj
1
(
k
)
exp
(
ⅈ
ϕ
j
1
)
∑
j
2
I
kj
2
(
k
)
exp
(
-
ⅈ
ϕ
j
2
)
∑
j
2
I
kj
2
(
k
)
exp
(
ⅈ
ϕ
j
2
)
∑
j
3
I
kj
3
(
k
)
exp
(
-
ⅈ
ϕ
j
3
)
∑
j
3
I
kj
3
(
k
)
exp
(
ⅈ
ϕ
j
3
)
)
=
(
m
0
(
N
1
+
N
2
+
N
3
)
m
-
1
∑
j
1
exp
(
-
ⅈ
ϕ
j
1
)
m
+
1
∑
j
1
exp
(
ⅈ
ϕ
j
1
)
m
-
1
∑
j
2
exp
(
-
ⅈ
ϕ
j
2
)
m
+
1
∑
j
2
exp
(
ⅈ
ϕ
j
2
)
m
-
1
∑
j
3
exp
(
-
ⅈ
ϕ
j
3
)
m
+
1
∑
j
3
exp
(
ⅈ
ϕ
j
3
)
m
0
∑
j
1
exp
(
-
ⅈ
ϕ
j
1
)
m
-
1
∑
j
1
exp
(
-
2
ⅈ
ϕ
j
1
)
m
+
1
N
1
0
0
0
0
m
0
∑
j
1
exp
(
ⅈ
ϕ
j
1
)
m
-
1
N
1
m
+
1
∑
j
1
exp
(
2
ⅈ
ϕ
j
1
)
0
0
0
0
m
0
∑
j
2
exp
(
-
ⅈ
ϕ
j
2
)
0
0
m
-
1
∑
j
2
exp
(
-
2
ⅈ
ϕ
j
2
)
M
+
1
N
2
0
0
m
0
∑
j
2
exp
(
ⅈ
ϕ
j
2
)
0
0
m
-
1
N
2
m
+
1
∑
j
2
exp
(
2
ⅈ
ϕ
j
2
)
0
0
m
0
∑
j
3
exp
(
-
ⅈ
ϕ
j
3
)
0
0
0
0
m
-
1
∑
j
3
exp
(
-
2
ⅈ
ϕ
j
3
)
m
+
1
N
3
m
0
∑
j
3
exp
(
ⅈ
ϕ
j
3
)
0
0
0
0
m
-
1
N
3
m
+
1
∑
j
3
exp
(
2
ⅈ
ϕ
j
3
)
)
(
O
k
(
k
)
P
k
(
k
)
O
k
1
(
k
-
K
)
P
k
(
k
)
O
k
1
(
k
+
K
)
P
k
(
k
)
O
k
2
(
k
-
K
)
P
k
(
k
)
O
k
2
(
k
+
K
)
P
k
(
k
)
O
k
3
(
k
-
K
)
P
k
(
k
)
O
k
3
(
k
+
K
)
P
k
(
k
)
)
(
2
)
where
Nd is the number of images in respective modulation directions,
m l is a modulated amplitude of the illuminating light,
φ j is a modulated phase of the jth one of images,
I kj (k) is a signal intensity of the jth one of images, and
P k (k) is a transfer function of the optical system.
5. A microscope device comprising:
a light source;
an illumination optical system that includes a diffraction grating which is disposed in a position that is optically conjugate with a sample and illuminates the sample with light sent from the light source and sinusoidally spatially modulated by the diffraction grating;
a modulation device that modulates the diffraction grating;
an image pickup means that picks up spatially modulated sample images from the sample; and
an image forming device that forms a sample image by applying the method of least squares which is shown by the following equation and in which complex numbers are considered,
(
∑
j
b
-
1
j
I
k
j
(
k
)
∑
j
b
0
j
I
k
j
(
k
)
∑
j
b
+
1
j
I
k
j
(
k
)
)
=
(
∑
j
b
-
1
j
2
∑
j
b
-
1
j
b
0
j
∑
j
b
-
1
j
b
+
1
j
∑
j
b
0
j
b
-
1
j
∑
j
b
0
j
2
∑
j
b
0
j
b
+
1
j
∑
j
b
+
1
j
b
-
1
j
∑
j
b
+
1
j
b
0
j
∑
j
b
+
1
j
2
)
(
O
k
(
k
-
K
)
P
k
(
k
)
O
k
(
k
)
P
k
(
k
)
O
k
(
k
+
K
)
P
k
(
k
)
)
(
3
)
where
b lj =m l exp(iφ j ),
m l is a modulated amplitude of the illuminating light,
φ j is a modulated phase of the jth one of images,
I kj (k) is a signal intensity of the jth one of images, and
P k (k) is a transfer function of the optical system,
with the amount by which the plurality of picked-up images are varied when these images are picked up for each of the spatial frequency components constituting a greater number of images than the number of spatially modulated components contained in the light emitted from the sample as a result of the illuminating light on the sample (picked up by the image pickup means) taken as a parameter.
6. The microscope device according to claim 5 , wherein the image pickup means picks up a plurality of images in which the phase of the illuminating light pattern on the sample is varied.
7. The microscope device according to claim 5 , wherein the image pickup means picks up a plurality of images in which the modulation amplitude of the illuminating light on the sample is varied.
8. The microscope device according to any one of claims 5 to 7 , wherein
the diffraction grating has grating patterns in two directions in the plane perpendicular to the optical axis, and
the modulation device moves the diffraction grating in the directions respectively perpendicular to these two directions.
9. The microscope device according to any one of claims 5 to 7 , wherein
the diffraction grating has a grating pattern in one direction in the plane perpendicular to the optical axis, and
using a plurality of images obtained by illuminating the sample with spatially modulated lights in three directions, one zero-order light modulated component, three +first-order light modulated components and three −first-order light modulated components are separated by applying the method of least squares, which is shown by the following equation in place of Equation (3)
(
∑
j
=
j
1
,
j
2
,
j
3
I
kj
(
k
)
∑
j
1
I
kj
1
(
k
)
exp
(
-
ⅈ
ϕ
j
1
)
∑
j
1
I
kj
1
(
k
)
exp
(
ⅈ
ϕ
j
1
)
∑
j
2
I
kj
2
(
k
)
exp
(
-
ⅈ
ϕ
j
2
)
∑
j
2
I
kj
2
(
k
)
exp
(
ⅈ
ϕ
j
2
)
∑
j
3
I
kj
3
(
k
)
exp
(
-
ⅈ
ϕ
j
3
)
∑
j
3
I
kj
3
(
k
)
exp
(
ⅈ
ϕ
j
3
)
)
=
(
m
0
(
N
1
+
N
2
+
N
3
)
m
-
1
∑
j
1
exp
(
-
ⅈ
ϕ
j
1
)
m
+
1
∑
j
1
exp
(
ⅈ
ϕ
j
1
)
m
-
1
∑
j
2
exp
(
-
ⅈ
ϕ
j
2
)
m
+
1
∑
j
2
exp
(
ⅈ
ϕ
j
2
)
m
-
1
∑
j
3
exp
(
-
ⅈ
ϕ
j
3
)
m
+
1
∑
j
3
exp
(
ⅈ
ϕ
j
3
)
m
0
∑
j
1
exp
(
-
ⅈ
ϕ
j
1
)
m
1
∑
j
1
exp
(
-
2
ⅈ
ϕ
j
1
)
m
+
1
N
1
0
0
0
0
m
0
∑
j
1
exp
(
ⅈ
ϕ
j
1
)
m
-
1
N
1
m
+
1
∑
j
1
exp
(
2
ⅈ
ϕ
j
1
)
0
0
0
0
m
0
∑
j
2
exp
(
-
ⅈ
ϕ
j
2
)
0
0
m
-
1
∑
j
2
exp
(
-
2
ⅈ
ϕ
j
2
)
M
+
1
N
2
0
0
m
0
∑
j
2
exp
(
ⅈ
ϕ
j
2
)
0
0
m
-
1
N
2
m
+
1
∑
j
2
exp
(
2
ⅈ
ϕ
j
2
)
0
0
m
0
∑
j
3
exp
(
-
ⅈ
ϕ
j
3
)
0
0
0
0
m
-
1
∑
j
3
exp
(
-
2
ⅈ
ϕ
j
3
)
m
+
1
N
3
m
0
∑
j
3
exp
(
ⅈ
ϕ
j
3
)
0
0
0
0
m
-
1
N
3
m
+
1
∑
j
3
exp
(
2
ⅈ
ϕ
j
3
)
)
(
O
k
(
k
)
P
k
(
k
)
O
k
1
(
k
-
K
)
P
k
(
k
)
O
k
1
(
k
+
K
)
P
k
(
k
)
O
k
2
(
k
-
K
)
P
k
(
k
)
O
k
2
(
k
+
K
)
P
k
(
k
)
O
k
3
(
k
-
K
)
P
k
(
k
)
O
k
3
(
k
+
K
)
P
k
(
k
)
)
(
4
)
where
Nd is the number of images in respective modulation directions,
m l is a modulated amplitude of the illuminating light,
φ j is a modulated phase of the jth one of images,
I kj (k) is a signal intensity of the jth one of images, and
P k (k) is a transfer function of the optical system.