Methods and apparatus for HILN characterization using convolutional neural network
A method of characterizing a serum and plasma portion of a specimen in regions occluded by one or more labels. The characterization may be used for determining Hemolysis (H), Icterus (I), and/or Lipemia (L), or Normal (N) of a serum or plasma portion of a specimen. The method includes capturing one or more images of a labeled specimen container including a serum or plasma portion, processing the one or more images with a convolutional neural network to provide a determination of Hemolysis (H), Icterus (I), and/or Lipemia (L), or Normal (N). In further embodiments, the convolutional neural network can provide N′-Class segmentation information. Quality check modules and testing apparatus adapted to carry out the method are described, as are other aspects.
1. A characterization method, comprising:
capturing multiple images of a specimen container including a serum or plasma portion of a specimen;
inputting image data from the multiple images to a convolutional neural network and processing the image data with the convolutional neural network; and
outputting from the convolutional neural network a classification of the serum or plasma portion as being one or more of hemolytic, icteric, lipemic, and normal; wherein:
the convolutional neural network comprises an architecture including at least two layers including convolution and pooling, and at least two additional fully convolution layers.
2. The method of claim 1 , wherein the capturing multiple images comprises capturing multiple images at different exposure times for each of multiple spectra.
3. The method of claim 1 , wherein the capturing the multiple images comprise providing different exposure times for each spectrum of red, green, and blue.
4. The method of claim 1 , wherein the multiple images are captured from multiple viewpoints and comprise multi-spectral, multi-exposure images for each viewpoint.
5. The method of claim 4 , wherein the specimen container includes one or more labels occluding at least part of one viewpoint.
6. The method of claim 1 , wherein the convolutional neural network architecture includes at least three layers including the convolution and pooling, and a deconvolutional layer and a SoftMax layer.
7. The method of claim 1 , wherein the convolutional neural network comprises:
multiple layers including convolution, LRN, and ReLU, fully convolutional layers, a deconvolution layer, and a SoftMax layer.
8. The method of claim 1 , wherein the classification of the serum or plasma portion comprises output options of N-classes for each of hemolytic, icteric, and lipemic.
9. The method of claim 1 , wherein the classification of the serum or plasma portion comprises output options of fine-grained indexes for each of hemolysis, icterus, and lipemia.
10. The method of claim 1 , wherein the capturing the multiple images comprises backlighting with light sources comprising one or more spectra of R, G, B, white light, infrared (IR), and near IR.
11. The method of claim 1 , wherein the image data from the multiple images comprises consolidated pixel or patch data from multiple exposures.
12. The method of claim 1 , wherein the convolutional neural network further outputs N′-class segmentation data.
13. The method of claim 1 , wherein the convolutional neural network further comprises:
a network branch including n-class output options for cap types.
14. A characterization method, comprising:
capturing multiple images of a specimen container including a serum or plasma portion of a specimen;
inputting image data from the multiple images to a convolutional neural network and processing the image data with the convolutional neural network; and
outputting from the convolutional neural network a classification of the serum or plasma portion as being one or more of hemolytic, icteric, lipemic, and normal;
wherein the convolutional neural network comprises an architecture including a deconvolution layer.
15. A characterization method, comprising:
capturing multiple images of a specimen container including a serum or plasma portion of a specimen;
inputting image data from the multiple images to a convolutional neural network and processing the image data with the convolutional neural network; and
outputting from the convolutional neural network a classification of the serum or plasma portion as being one or more of hemolytic, icteric, lipemic, and normal;
wherein the convolutional neural network comprises an architecture including a SoftMax layer.
16. A characterization method, comprising:
capturing multiple images of a specimen container including a serum or plasma portion of a specimen;
inputting image data from the multiple images to a convolutional neural network and processing the image data with the convolutional neural network; and
outputting from the convolutional neural network a classification of the serum or plasma portion as being one or more of hemolytic, icteric, lipemic, and normal;
wherein the classification of the serum or plasma portion comprises outputting a fine-grained index for one or more of hemolysis, icterus, and lipemia.
17. A characterization method, comprising:
capturing multiple images of a specimen container including a serum or plasma portion of a specimen;
inputting image data from the multiple images to a convolutional neural network and processing the image data with the convolutional neural network; and
outputting from the convolutional neural network a classification of the serum or plasma portion as being one or more of hemolytic, icteric, lipemic, and normal;
wherein the convolutional neural network comprises:
a first network branch including N′-class segmentation outputs, and
a second network branch including N-class output options classes for each of hemolytic, icteric, and lipemic.
18. A quality check module for pre-screening a specimen for presence of an interferent, comprising:
an image capture device configured to capture multiple images of a specimen container containing a serum or plasma portion of a specimen; and
a computer coupled to the image capture device, the computer configured and capable of being operated to:
input image data from the multiple images to a convolutional neural network and process the image data with the convolutional neural network, the convolutional neural network comprising an architecture that includes a deconvolution layer, and
output from the convolutional neural network a classification of the serum or plasma portion as being one or more of hemolytic, icteric, lipemic, and normal.
19. A specimen testing apparatus, comprising:
a track;
a carrier moveable on the track and configured to contain a specimen container containing a serum or plasma portion of a specimen;
image capture devices arranged around the track and configured to capture multiple images of a specimen container and the serum or plasma portion of the specimen from multiple viewpoints; and
a computer coupled to the image capture devices, the computer configured and capable of being operated to:
input image data from the multiple images to a convolutional neural network and process the image data with the convolutional neural network, the convolutional neural network comprising an architecture that includes a SoftMax layer, and
output from the convolutional neural network a classification of the serum or plasma portion as being one or more of hemolytic, icteric, lipemic, and normal.