Segmenting 3D intracellular structures in microscopy images using an iterative deep learning workflow that incorporates human contributions
A facility for identifying the boundaries of 3-dimensional structures in 3-dimensional images is described. For each of multiple 3-dimensional images, the facility receives results of a first attempt to identify boundaries of structures in the 3-dimensional image, and causes the results of the first attempt to be presented to a person. For each of a number of 3-dimensional images, the facility receives input generated by the person providing feedback on the results of the first attempt. The facility then uses the following to train a deep-learning network to identify boundaries of 3-dimensional structures in 3-dimensional images: at least a portion of the plurality of 3-dimensional images, at least a portion of the received results, and at least a portion of provided feedback.
1. A method in a computing system for identifying boundaries of 3-dimensional structures in 3-dimensional images, comprising:
for each of a plurality of 3-dimensional images,
receiving results of a first attempt to identify boundaries of structures in the 3-dimensional image by applying a sequence of one or more procedural segmentation measures;
causing the results of the first attempt to be presented to a person;
receiving input generated by the person providing feedback on the results of the first attempt;
using the following to train a deep-learning network to identify boundaries of 3-dimensional structures in 3-dimensional images: at least a portion of the plurality of 3-dimensional images, at least a portion of the received results, and at least a portion of provided feedback;
receiving a distinguished 3-dimensional image not among the plurality of 3-dimensional images;
applying the trained deep-learning network to the distinguished 3-dimensional image to obtain a segmentation result for the distinguished 3-dimensional image; and
storing the obtained segmentation result for the distinguished 3-dimensional image, wherein the feedback approves the results of the first attempt for a first proper subset of the plurality of 3-dimensional images and rejects the results of the first attempt for a second proper subset of the plurality of 3-dimensional images.
2. The method of claim 1 wherein the structures are organelles in a biological cell.
3. The method of claim 1 wherein, for each of at least one of the plurality of 3-dimensional images, the feedback subdivides the results into two or more regions in which structure boundaries should be identified independently.
4. The method of claim 1 wherein the deep-learning network is one or more artificial neural networks.
5. The method of claim 1 wherein the deep-learning network is a U-net variant.
6. The method of claim 1 wherein each 3-dimensional image is represented by a stack of 2-dimensional images.
7. A non-transitory computer-readable medium having contents configured to cause a computing system to perform a method in a computing system for identifying boundaries of 3-dimensional structures in 3-dimensional images, the method comprising:
for each of a plurality of 3-dimensional images,
receiving results of a first attempt to identify boundaries of structures in the 3-dimensional image by applying a sequence of one or more procedural segmentation measures;
causing the results of the first attempt to be presented to a person;
receiving input generated by the person providing feedback on the results of the first attempt;
using the following to train a deep-learning network to identify boundaries of 3-dimensional structures in 3-dimensional images: at least a portion of the plurality of 3-dimensional images, at least a portion of the received results, and at least a portion of provided feedback;
receiving a distinguished 3-dimensional image not among the plurality of 3-dimensional images;
applying the trained deep-learning network to the distinguished 3-dimensional image to obtain a segmentation result for the distinguished 3-dimensional image; and
storing the obtained segmentation result for the distinguished 3-dimensional image, wherein the feedback approves the results of the first attempt for a first proper subset of the plurality of 3-dimensional images and rejects the results of the first attempt for a second proper subset of the plurality of 3-dimensional images.
8. The non-transitory computer-readable medium of claim 7 wherein the structures are organelles in a biological cell.
9. The non-transitory computer-readable medium of claim 7 wherein, for each of at least one of the plurality of 3-dimensional images, the feedback subdivides the results into two or more regions in which structure boundaries should be identified independently.
10. The non-transitory computer-readable media medium of claim 7 wherein the deep-learning network is a sequence of artificial neural networks.
11. The non-transitory computer-readable medium of claim 7 wherein the deep-learning network is a U-net variant.
12. The non-transitory computer-readable medium of claim 7 wherein each 3-dimensional image is represented by a stack of 2-dimensional images.
13. A computing system, comprising:
at least one processor; and
a memory having contents configured to cause the at least one processor to perform a method for identifying boundaries of 3-dimensional structures in 3-dimensional images, the method comprising:
for each of a plurality of 3-dimensional images,
receiving results of a first attempt to identify boundaries of structures in the 3-dimensional image by applying a sequence of one or more procedural segmentation measures;
causing the results of the first attempt to be presented to a person;
receiving input generated by the person providing feedback on the results of the first attempt;
using the following to train a deep-learning network to identify boundaries of 3-dimensional structures in 3-dimensional images: at least a portion of the plurality of 3-dimensional images, at least a portion of the received results, and at least a portion of provided feedback;
receiving a distinguished 3-dimensional image not among the plurality of 3-dimensional images;
applying the trained deep-learning network to the distinguished 3-dimensional image to obtain a segmentation result for the distinguished 3-dimensional image; and
storing the obtained segmentation result for the distinguished 3-dimensional image,
wherein the feedback approves the results of the first attempt for a first proper subset of the plurality of 3-dimensional images and rejects the results of the first attempt for a second proper subset of the plurality of 3-dimensional images.
14. The computing system of claim 13 wherein the structures are organelles in a biological cell.
15. The computing system of claim 13 wherein, for each of at least one of the plurality of 3-dimensional images, the feedback subdivides the results into two or more regions in which structure boundaries should be identified independently.
16. The computing system of claim 13 wherein the deep-learning network is one or more artificial neural networks.
17. The computing system of claim 13 wherein the deep-learning network is a U-net variant.
18. The computing system of claim 13 wherein each 3-dimensional image is represented by a stack of 2-dimensional images.