Granted Patent B2

US 12,700,249 · App. 18/348,188 · Granted Aug 4, 2026

Inventors: Hirofumi Kobayashi (Los Angeles, CA); Yair Rivenson (Los Angeles, CA); Luis Carvalho (Valencia, CA)

Assignee: Pictor Labs, Inc.

G06V20/695G06T5/20G06T5/50G06T5/70G06V10/30G06V10/82G06T2207/20081G06T2207/20084G06T2207/30024

Abstract

Claims (41)

1 . A method of refining virtually-stained image annotations, the method comprising:

separating an image of a stained tissue section into an image of a counterstain and an image of immunostaining visual signals via linear transformation;

identifying and segmenting nuclei in the image of the counterstain to produce an image of a refined counterstain with single-cell segments;

comparing immunostaining visual signals in the single-cell segments of the image of the refined counterstain to a user-defined threshold to determine true positive cells, wherein the comparing comprises summing an immunostaining visual signal intensity within each single-cell segment;

refining the image of the immunostaining visual signals to produce an image of refined immunostaining visual signals, the refining based on the stained tissue section protein localization or morphology and the determined true positive cells; and

overlaying the image of the counterstain and the image of refined immunostaining visual signals to produce a refined virtually-stained image annotation.

2 . The method of claim 1 , wherein the counterstain is selected from the group consisting of hematoxylin, Hoechst, and 4′,6-diamidino-2-phenylindole (DAPI).

3 . The method of claim 1 , wherein refining the image of the immunostaining visual signals includes blurring, using a Gaussian filter, to simulate membrane, cytoplasmic, or nuclei staining.

4 . The method of claim 1 , further comprising training a neural network to produce refined virtually-stained image annotations, an image of an unstained tissue section as an input into the neural network.

5 . The method of claim 1 , wherein overlaying the image of the refined counterstain the image of refined immunostaining visual signals comprises:

concatenating the image of the refined counterstain and the image of the immunostaining visual signals at a third dimension to produce a concatenation; and

performing linear transformation to convert the concatenation to an image native color space for display purposes.

6 . The method of claim 1 , wherein comparing immunostaining visual signals to a user-defined threshold in the single-cell segments of the image of the refined counterstain further comprises:

totaling a number of pixels in the image of the immunostaining visual signals that have an immunostaining visual signal higher than 0 within each single-cell segment.

7 . The method of claim 1 , wherein comparing immunostaining visual signals to a user-defined threshold in the single-cell segments of the image of the refined counterstain further comprises:

analyzing the maximum intensity of immunostaining visual signal within each single-cell segment.

8 . The method of claim 1 , wherein comparing immunostaining visual signals to a user-defined threshold in the single-cell segments of the image of the refined counterstain further comprises:

defining true positive cells by dividing a total amount of immunostaining visual signal in a cell region by an area of segmentation region of the cell.

9 . The method of claim 1 , wherein segmenting nuclei in the image of the counterstain includes dilating the nuclei to include cytoplasmic area.

10 . The method of claim 1 , wherein the immunostaining visual signal is selected from the group consisting of fluorescence tags, fluorescence expressing proteins, and 3,3′-diaminobenzidine (DAB) stains.

11 . The method of claim 1 , wherein comparing the immunostaining visual signals to a user-defined threshold in the single-cell segments of the image of the refined counterstain further comprises:

defining true positive cells by dividing a total amount of the immunostaining visual signal in a cell region by an area of segmentation region of the cell.

12 . A method of refining virtually-stained image annotations, the method comprising:

separating an image of a stained tissue section into an image of a counterstain and an image of immunostaining visual signals via linear transformation;

identifying and segmenting nuclei in the image of the counterstain to produce an image of a refined counterstain with single-cell segments;

comparing immunostaining visual signals in the single-cell segments of the image of the refined counterstain to a user-defined threshold to determine true positive cells, wherein the comparing comprises totaling a number of pixels in the image of the immunostaining visual signals that have an immunostaining visual signal higher than 0 within each single-cell segment;

refining the image of the immunostaining visual signals to produce an image of refined immunostaining visual signals, the image refining based on the stained tissue section protein localization or morphology and the determined true positive cells; and

training a neural network to produce refined virtually-stained image annotations, an image of an unstained tissue section as an input into the neural network.

13 . The method of claim 12 , wherein the immunostaining visual signal is selected from the group consisting of fluorescence tags, fluorescence expressing proteins, and 3,3′-diaminobenzidine (DAB) stains.

14 . The method of claim 12 , wherein the counterstain is selected from the group consisting of hematoxylin, Hoechst, and 4′,6-diamidino-2-phenylindole (DAPI).

15 . The method of claim 12 , wherein refining the image of the immunostaining visual signals includes burring, using a Gaussian filter, to simulate membrane, cytoplasmic, or nuclei staining.

16 . The method of claim 12 , wherein comparing the immunostaining visual signals to a user-defined threshold in the single-cell segments of the image of the refined counterstain further comprises:

summing the immunostaining visual signal intensities within each single-cell segment.

17 . The method of claim 12 , wherein comparing the immunostaining visual signals to a user-defined threshold in the single-cell segments of the image of the refined counterstain further comprises:

analyzing a maximum intensity of the immunostaining visual signals within each single-cell segment.

18 . A method of refining virtually-stained image annotations, the method comprising:

separating an image of a stained tissue section via linear transformation into two separate stained channels, the two separate stained channels including an image of a counterstain and an image of 3,3′-diaminobenzidine (DAB) stain;

segmenting nuclei in the image of the counterstain to produce an image of single-cell segments;

comparing DAB signals of the image of single-cell segments to a threshold to determine true positive cells, wherein the comparing comprises analyzing a maximum intensity of the DAB signals within each single-cell segment;

refining the image of the DAB stain depending on the determined true positive cells, producing an image of refined immunostaining visual signals; and

training a neural network to produce refined virtually-stained image annotations, the refined image of the immunostaining visual signals as an input into the neural network.

Assignments (2)

Continuity (1)

References Cited (258)