IP Library Granted Patent US 10,255,693
Granted Patent B2
US 10,255,693 · App. 15/585,045 · Granted Apr 9, 2019

Machine learning classification and training for digital microscopy images

Inventor: Richard Boyd Smith (Orem, UT)
Assignee: Techcyte, Inc.
G06T7/97G06K9/6267G06T2207/10056G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,255,693
App. No.
15/585,045
Granted
Apr 9, 2019
Kind
B2
Abstract

Systems, methods, and devices for training models or algorithms for classifying or detecting particles or materials in microscopy images are disclosed. A method includes receiving a plurality of microscopy images of a specimen and a classification for the specimen. The plurality of microscopy images includes a first image captured at a first magnification and a second image captured at the first magnification with a different focus than the first image. The method includes training a machine learning model or algorithm using the plurality of images, wherein the first image and the second image are provided with one or more labels indicating the classification.

Claims (51)

1. A method comprising:

receiving a specimen sample disposed on a slide, the slide including a slide label indicating a classification for the specimen;

imaging the specimen sample disposed on the slide;

receiving a plurality of microscopy images of the specimen sample and classification for the specimen sample, the plurality of microscopy images comprising a first image captured at a first magnification and a second image captured at the first magnification with a different focal depth than the first image, wherein the focal depth for the second image overlaps with a focal depth of the first image;

training a machine learning model or algorithm using the plurality of images, wherein the training further includes providing the first image and the second image with overlapping focal depths to the machine learning model or algorithm with the classification of the specimen sample;

labeling the plurality of microscopy images with image labels indicating the classification of the sample specimen; and

associating the slide label with the image labels on the plurality of microscopy images, the slide label further identifying a region of the slide from which the plurality of microscopy images were imaged.

2. The method of claim 1 , wherein receiving the plurality of microscopy images of the sample specimen comprises capturing the plurality of images of the sample specimen, wherein:

capturing the first image comprises capturing at the first magnification with a first focal plane; and

capturing the second image comprises capturing at the first magnification with a second focal plane different than the first focal plane.

3. The method of claim 2 , wherein capturing the plurality of microscopy images of the specimen sample comprises capturing microscopy images of a specimen sample by:

capturing a plurality of images with different focal planes of a first region of the slide; and

capturing a plurality of images with different focal planes of a second one or more regions of the slide, such that each captured region of the slide has a plurality of corresponding images with different focal planes.

4. The method of claim 3 , wherein each captured region of the slide has a plurality of corresponding images with different focal planes comprising two or more different focal planes.

5. The method of claim 1 , wherein training the machine learning model or algorithm comprises:

providing the first image with the classification as ground truth; and

providing the second image with the classification as ground truth.

6. The method of claim 1 , further comprising determining the classification, wherein determining the classification comprises determining a classification based on data not included in the plurality of microscopy images.

7. The method of claim 1 , further comprising determining the classification, wherein determining the classification comprises determining a classification based on a combination of data found in two or more of the plurality of microscopy images.

8. The method of claim 1 , wherein the machine learning model or algorithm comprises a deep neural network, wherein training the deep neural network comprises training using a back-propagation algorithm.

9. The method of claim 1 , wherein the specimen comprises a particle, wherein the classification comprises a classification for the particle, and wherein training comprises training the machine learning model or algorithm to determine a classification for a particle.

10. A system for training a neural network, the system comprising:

one or more processors; and

non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to:

image a specimen sample disposed on a slide, the slide including a slide label indicating a classification for the specimen sample;

receive a plurality of microscopy images of the specimen sample and the classification for the specimen sample, the plurality of microscopy images comprising a first image captured at a first magnification and a second image captured at the first magnification with a different focal depth than the first image, wherein the focal depth for the second image overlaps with a focal depth of the first image; and

train a machine learning model or algorithm using the plurality of images, wherein the training further includes providing the first image and the second image with overlapping focal depths to the machine learning model or algorithm with the classification of the specimen sample;

labeling the plurality of microscopy images with image labels indicating the classification of the sample specimen; and

associating the slide label with the image labels on the plurality of microscopy images, the slide label further identifying a region of the slide from which the plurality of microscopy images were imaged.

11. The system of claim 10 , wherein receiving the plurality of microscopy images of the sample specimen comprises capturing the plurality of images of the sample specimen, wherein the instructions cause the one or more processors to control an imaging system to:

capture the first image at the first magnification with a first focal plane; and

capture the second image at the first magnification with a second focal plane different than the first focal plane.

12. The system of claim 11 , wherein capturing the plurality of microscopy images of the specimen sample comprises capturing microscopy images of a specimen sample by:

capturing a plurality of images with different focal planes of a first region of the slide; and

capturing a plurality of images with different focal planes of a second one or more regions of the slide, such that each captured region of the slide has a plurality of corresponding images with different focal planes.

13. The system of claim 12 , wherein each captured region of the slide has a plurality of corresponding images with different focal planes comprising two or more different focal planes.

14. The system of claim 10 , wherein the instructions cause the one or more processors to train the machine learning model or algorithm by:

providing the first image with the classification as ground truth; and

providing the second image with the classification as ground truth.

15. The system of claim 10 , wherein the instructions cause the one or more processors to determine the classification, wherein determining the classification comprises obtaining classification information based on data not included in the plurality of microscopy images.

16. The system of claim 10 , wherein training the machine learning model or algorithm comprises training a deep neural network, wherein training the deep neural network comprises training using a back-propagation algorithm.

17. Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to:

obtain microscopy images of a specimen sample disposed on a slide having a slide label indicating a classification for the specimen sample, wherein obtaining microscopy image comprises, for one or more regions of the sample:

capturing a first image of the specimen sample with a first focal depth at a first magnification;

capturing one or more additional images with one or more additional focal depths at the first magnification, wherein the first focal depth overlaps at least one of the one or more additional focal depths;

training a neural network using the first image and the one or more additional images, wherein the training further includes providing the first image and the one or more additional images with the overlapping focal depths to the neural network with the classification of the specimen sample;

labeling the first image and the one or more additional images with image labels indicating the classification of the sample specimen; and

associating the slide label with the image labels on the first image and the one or more additional images, the slide label for the first image including ground truth for the first image and the slide label for the one or more additional images identifies a same region and same specimen captured in the first image and the one or more additional images.

18. The computer readable storage media of claim 17 , wherein the instructions cause the one or more processors to train the machine learning model or algorithm by training a deep neural network, wherein training the deep neural network comprises training using a back-propagation algorithm.

19. The computer readable storage media of claim 17 , wherein the instructions cause the one or more processors to determine the classification, wherein determining the classification comprises obtaining classification information based on data not included in the plurality of microscopy images.

20. The computer readable storage media of claim 17 , wherein each of the one or more regions has a plurality of corresponding images with different focal planes comprising two or more different focal planes.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 3, 2017
From: SMITH, RICHARD BOYD
To: TECHCYTE, INC.
Reel/Frame 042224/0391 →
Continuity (1)
Related Publication 20180322660A1 · Nov 8, 2018
Cited By (4)
US 12,300,378 US 12,475,564 US 12,561,402 US 12,689,822