Analyzing complex single molecule emission patterns with deep learning
A fluorescent single molecule emitter simultaneously transmits its identity, location, and cellular context through its emission patterns. A deep neural network (DNN) performs multiplexed single-molecule analysis to enable retrieving such information with high accuracy. The DNN can extract three-dimensional molecule location, orientation, and wavefront distortion with precision approaching the theoretical limit of information content of the image which will allow multiplexed measurements through the emission patterns of a single molecule.
1 . A computer-implemented method for training a neural network for use in extracting physical information from an image of point spread function (PSF) emission patterns of illuminated single molecules, the emission patterns corresponding to ground truth parameters θ tn indicative of the physical information of the molecule, the method comprising:
(a) acquiring a dataset of a plurality of N×N pixel images A n of the PSF emission patterns and providing the dataset to a processor;
(b) forward propagating the dataset through a neural network, with the processor, comprising a plurality of convolutional and residual layers l and a plurality of training parameters w l of the layers l, to generate an output vector of a number of physical parameters {circumflex over (θ)} tn in a k th iteration through the neural network;
(c) determining with the processor a measure of the information content of the dataset regarding the number of physical parameters;
(d) comparing, with the processor, the output physical parameters {circumflex over (θ)} tn to the ground truth parameters θ tn by calculating a mean squared error (MSE) weighted by the measure of information content based on the equation
E
θ
ˆ
=
1
NT
∑
n
=
1
N
∑
t
=
1
T
(
θ
ˆ
tn
-
θ
tn
)
2
Measure
θ
tn
,
where N is the number of images A n , T is the size of the output vector for each image, and Measure θ tn is the value for parameter θ t of image A n of the information content of the dataset;
(e) updating, with the processor, the parameters w l of the layers l of the neural network based on the derivative of the CRLB-weighted MSE according to the equation
w
k
+
1
l
=
w
k
l
-
η
M
∑
n
=
1
M
∂
E
θ
^
n
,
k
∂
w
n
,
k
l
,
where l=0,1,2, . . . L denotes the layers in the neural network, η is the learning rate, k is the iteration number, and M is the number of images A n ; and
(f) repeating steps (c)-(e) for successive iterations until the error reaches a predetermined limit, after which the parameters w l of the layers l of the neural network are assigned, with the processor, the updated values of the last iteration for subsequently using the neural network, with the processor, to identify unknown physical parameters from the PSF of a new image of a single molecule.
2 . The method of claim 1 , wherein the physical parameters include one or more of axial (z) position, lateral (x, y) position, wavefront distortion and dipole orientation of the single molecule.
3 . The method of claim 2 , wherein the output vector includes at least two physical parameters.
4 . The method of claim 1 , wherein the plurality of layers of the neural network includes a fully connected layer between the output and the last one of the convolutional and residual layers.
5 . The method of claim 4 , wherein the plurality of layers of the neural network includes a Hard Tanh layer between the fully connected layer and the output.
6 . The method of claim 1 , wherein each residual layer includes a PreLU (Parametric Rectified Linear Unit) activation function at the output of each residual layer.
7 . The method of claim 1 , wherein the measure of information content is a Cramér-Rao lower bound (CRLB) for the dataset.
8 . The method of claim 1 , further comprising:
dividing the acquired dataset into a training dataset and a validation dataset;
forward propagating both datasets through said neural network;
for both datasets, calculating the mean squared error weighted by the information content; and
comparing the error for the validation dataset to said predetermined limit.
9 . The method of claim 8 , wherein the predetermined limit is when the error for the validation dataset stops decreasing between successive iterations of Steps (c)-(e).
10 . The method of claim 1 , wherein after acquiring the dataset, the dataset is batch normalized by dividing each pixel value within the dataset by the maximum pixel value of the image.
11 . The method of claim 1 , further comprising the step of shuffling the order of the images within the dataset prior to forward propagating the dataset through the neural network for each of the successive iterations.