Identifying microorganisms using three-dimensional quantitative phase imaging
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying the predicted type of one or more microorganisms. In one aspect, a system comprises a phase-contrast microscope and a microorganism classification system. The phase-contrast microscope is configured to generate a three-dimensional quantitative phase image of one or more microorganisms. The microorganism classification system is configured to process the three-dimensional quantitative phase image using a neural network to generate a neural network output characterizing the microorganisms, and thereafter identify the predicted type of the microorganisms using the neural network output.
1. A method comprising:
using a phase-contrast microscope to generate one or more three-dimensional (3D) quantitative phase images of one or more microscopic entities,
wherein each of the 3D quantitative phase images comprises a respective 3D representation of the one or more microscopic entities as a 3D refractive index tomogram represented as a 3D matrix of numerical values, where each component of the 3D matrix corresponds to a respective 3D spatial position and defines a refractive index at the corresponding 3D spatial position; and
processing the one or more 3D quantitative phase images using a neural network, wherein:
the neural network is a convolutional neural network comprising one or more 3D convolutional layers, wherein each of the 3D convolutional layers performs operations comprising:
receiving a layer input that comprises one or more 3D matrices of features derived from the one or more 3D quantitative phase images input to the neural network; and
processing the layer input to generate a layer output, comprising convolving one or more 3D convolutional filters with the one or more 3D matrices included in the layer input; and
the neural network is configured to process the one or more three-dimensional quantitative phase images in accordance with trained parameter values of the neural network to generate a neural network output characterizing the one or more microscopic entities; and
classifying the one or more microscopic entities using the neural network output.
2. The method of claim 1 , wherein one or more of the microscopic entities are microscopic organisms.
3. The method of claim 2 , wherein one or more of the microscopic entities are multi-cellular organisms.
4. The method of claim 1 , wherein each of the one or more 3D quantitative phase images is captured at a different time point.
5. The method of claim 1 , wherein:
the neural network output comprises a respective probability value for each of a predetermined number of possible classes; and
the probability value for a given class indicates a likelihood that the one or more microscopic entities are included in the given class.
6. The method of claim 5 , wherein each class represents a respective type of microscopic entity.
7. The method of claim 6 , wherein a type of a microscopic entity characterizes one or more of: a genus of the microscopic entity, a species of the microscopic entity, a strain of the microscopic entity, a gram-stainability of the microscopic entity, a metabolism of the microscopic entity, a morphology of the microscopic entity, or a motility of the microscopic entity.
8. The method of claim 5 , wherein classifying the one or more microscopic entities using the neural network output comprises:
classifying the one or more microscopic entities as being included in a class associated with a highest probability value in the neural network output.
9. The method of claim 1 , wherein the one or more microscopic entities are bacteria, or viruses, or fungi, or parasites, or microalgae.
10. The method of claim 9 , wherein one or more of the microscopic entities are bacteria.
11. The method of claim 10 , wherein the bacteria are present in a blood sample of a patient.
12. The method of claim 11 , further comprising administering an antibiotic therapy to the patient based on the classification of the bacteria.
13. The method of claim 1 , wherein using a phase-contrast microscope to generate one or more three-dimensional quantitative phase images of the one or more microscopic entities comprises:
using the phase-contrast microscope to generate phase and amplitude images of the one or more microscopic entities at each of a plurality of illumination angles; and
reconstructing a three-dimensional refractive index tomogram using the phase and amplitude images.
14. The method of claim 1 , wherein the method takes at most one hour.
15. A system comprising:
a phase-contrast microscope that is configured to generate one or more three-dimensional quantitative phase image of one or more microscopic entities,
wherein each of the 3D quantitative phase images comprises a respective 3D representation of the one or more microscopic entities as a 3D refractive index tomogram represented as a 3D matrix of numerical values, where each component of the 3D matrix corresponds to a respective 3D spatial position and defines a refractive index at the corresponding 3D spatial position; and
a microscopic entity classification system that is configured to perform operations comprising:
processing the one or more 3D quantitative phase images using a neural network, wherein:
the neural network is a convolutional neural network comprising one or more 3D convolutional layers, wherein each of the 3D convolutional layers performs operations comprising:
receiving a layer input that comprises one or more 3D matrices of features derived from the one or more 3D quantitative phase images input to the neural network; and
processing the layer input to generate a layer output, comprising convolving one or more 3D convolutional filters with the one or more 3D matrices included in the layer input; and
the neural network is configured to process the one or more three-dimensional quantitative phase images in accordance with trained parameter values of the neural network to generate a neural network output characterizing the one or more microscopic entities; and
classifying the one or more microscopic entities using the neural network output.
16. The system of claim 15 , wherein one or more of the microscopic entities are microscopic organisms.
17. The system of claim 16 , wherein one or more of the microscopic entities are multi-cellular organisms.
18. The system of claim 15 , wherein each of the one or more 3D quantitative phase images is captured at a different time point.
19. The system of claim 15 , wherein:
the neural network output comprises a respective probability value for each of a predetermined number of possible classes; and
the probability value for a given class indicates a likelihood that the one or more microscopic entities are included in the given class.
20. The system of claim 19 , wherein each class represents a respective type of microscopic entity.