IP Library › Granted Patent US 12,333,726
Granted Patent B2
US 12,333,726 · App. 17/405,843 · Granted Jun 17, 2025

Specimen analysis method and image processing method

Inventors: Maki Hirai (Kyoto, JP); Hiroshi Ogi (Kyoto, JP); Shunta Ishihara (Kyoto, JP); Takahiro Tsujikawa (Kyoto, JP); Kyoko Itoh (Kyoto, JP)
Assignees: SCREEN Holdings Co., Ltd.; Kyoto Prefectural Public University Corporation
G06T7/0014G01N15/1429G06T7/187G06T7/30G06V10/22G01N15/01G01N2015/1006G01N2015/1497G06T2207/10056G06T2207/30024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,333,726
App. No.
17/405,843
Granted
Jun 17, 2025
Kind
B2
Abstract

An image processing using a plurality of specimen images obtained by successively applying a plurality of types of staining to a specimen to be evaluated and imaging the specimen after staining for at least two types of staining is performed. The image processing comprises: extracting an inner region corresponding to inward of a cell membrane of a single cell in the specimen based on at least one of the specimen images; specifying a cell region corresponding to an individual cell included in the specimen by expanding the inner region outwardly; and performing image cytometry for the cell region based on the specimen images. In analyzing a pathological specimen on a cell-by-cell basis, it is possible to deal with multiple immunostaining and specify the positions of individual cells and evaluate each cell separately even when a cell density in the specimen is high.

Claims (28)

1. A specimen analysis method, comprising:

obtaining a plurality of specimen images by successively applying a plurality of types of staining to a specimen to be evaluated and imaging the specimen after staining for at least two types of staining;

extracting an inner region being a closed region surrounded by a cell membrane and corresponding to inward of the cell membrane of a cell in the specimen by inputting at least one of the specimen images to a learning model constructed by machine learning in advance and obtaining an output image;

separating each extracted inner region as an individual cell included in the specimen from other inner region of other cells;

specifying a cell region corresponding to the individual cell by expanding the extracted inner region outwardly, and

performing image cytometry for the specified cell region based on the specimen images,

wherein the inner region is extracted based on the plurality of specimen images aligned with each other by executing alignment utilizing positions corresponding to each other among the plurality of specimen images,

wherein, for specifying the cell region:

an image processing which expands each point on a contour of the extracted inner region outwardly is performed until an expansion amount at the point reaches a predetermined specified value or a point on an expanded contour of the inner region contacts the other expanded inner region; and

a total region of the inner region and an expanded region by the image processing is specified as the cell region, and

wherein the inner region is extracted based on a combination of a plurality of the specimen images of different types of staining including a staining in which the cell membrane is selectively stained.

2. The specimen analysis method according to claim 1 , wherein the specimen is a tissue section having a thickness of less than 4 μm.

3. The specimen analysis method according to claim 1 , wherein the specimen is a tissue section having a thickness equal to or less than 2 μm.

4. The specimen analysis method according to claim 1 , wherein

the inner region is extracted from a specimen image captured after a staining in which a biological material present only inside of the cell membrane is selectively stained out of the plurality of types of staining.

5. The specimen analysis method according to claim 1 , wherein the specimen is decolorized before next staining after the imaging.

6. An image processing method using a plurality of specimen images obtained by successively applying a plurality of types of staining to a specimen to be evaluated and imaging the specimen after staining for at least two types of staining, the image processing method comprising:

extracting an inner region being a closed region surrounded by a cell membrane and corresponding to inward of the cell membrane of a cell in the specimen by inputting at least one of the specimen images to a learning model constructed by machine learning in advance and obtaining an output image;

separating each extracted inner region as an individual cell included in the specimen from other inner region of other cells;

specifying a cell region corresponding to the individual cell by expanding the extracted inner region outwardly, and

performing image cytometry for the specified cell region based on the specimen images,

wherein the inner region is extracted based on the plurality of specimen images aligned with each other by executing alignment utilizing positions corresponding to each other among the plurality of specimen images,

wherein for specifying the cell region:

an image processing which expands each point on a contour of the extracted inner region outwardly is performed until an expansion amount at the point reaches a predetermined specified value or a point on an expanded contour of the inner region contacts the other expanded inner region; and

a total region of the inner region and an expanded region by the image processing is specified as the cell region, and

wherein the inner region is extracted based on a combination of a plurality of the specimen images of different types of staining including a staining in which the cell membrane is selectively stained.

7. The specimen analysis method according to claim 1 , wherein a closed region surrounded by the cell membrane and not including the cell membrane is extracted as the inner region.

8. The image processing method according to claim 6 , wherein a closed region surrounded by the cell membrane and not including the cell membrane is extracted as the inner region.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2021
From: HIRAI, MAKI; OGI, HIROSHI; ISHIHARA, SHUNTA; TSUJIKAWA, TAKAHIRO; ITOH, KYOKO
To: SCREEN HOLDINGS CO., LTD.; KYOTO PREFECTURAL PUBLIC UNIVERSITY CORPORATION
Reel/Frame 057219/0591 →
Priority Claims (1)
JP 2020-141765 · Aug 25, 2020 · national
Continuity (1)
Related Publication 20220067938A1 · Mar 3, 2022
References Cited (44)
US 7372985B2 · So · 2008 [cited by examiner]
US 7756305B2 · Price · 2010 [cited by examiner]
US 8600142B2 · Berndt · 2013 [cited by examiner]
US 9224193B2 · Tsujimoto · 2015 [cited by examiner]
US 9934571B2 · Ozaki · 2018 [cited by examiner]
US 10890576B2 · Mimura · 2021 [cited by examiner]
US 11074692B2 · Sekiguchi · 2021 [cited by examiner]
US 11222415B2 · Ozcan · 2022 [cited by examiner]
US 11436718B2 · Yoshida · 2022 [cited by examiner]
US 11482320B2 · Klaiman · 2022 [cited by examiner]
US 11842556B2 · Sekiguchi · 2023 [cited by examiner]
US 20060257013A1 · Ramm et al. · 2006 [cited by applicant]
US 20090297015A1 · Jetzek · 2009 [cited by applicant]
US 20130108139A1 · Binnig et al. · 2013 [cited by applicant]
US 20150125953A1 · Hashino · 2015 [cited by examiner]
US 20150138334A1 · Usuba · 2015 [cited by examiner]
US 20160216209A1 · Okada et al. · 2016 [cited by applicant]
US 20170103521A1 · Chukka et al. · 2017 [cited by applicant]
US 20170160171A1 · Tsujikawa · 2017 [cited by examiner]
US 20190042958A1 · Letterie et al. · 2019 [cited by applicant]
US 20190064037A1 · Boyden et al. · 2019 [cited by applicant]
US 20200090330A1 · Chefd'hotel · 2020 [cited by examiner]
CN 111260677A · 2020 [cited by applicant]
EP 3232402A1 · 2017 [cited by applicant]
EP 3611695A1 · 2020 [cited by applicant]
JP 2005525550A · 2005 [cited by applicant]
JP 2013113818A · 2013 [cited by examiner]
JP 2017068308A · 2017 [cited by examiner]
JP 2019509057A · 2019 [cited by applicant]
JP 2019511914A · 2019 [cited by applicant]
WO 2015045962A1 · 2015 [cited by applicant]
WO WO2016076104A1 · 2016 [cited by examiner]
WO WO2018198253A1 · 2018 [cited by examiner]
WO WO2019188647A1 · 2019 [cited by examiner]
WO WO2019225506A1 · 2019 [cited by examiner]
WO WO2020145605A1 · 2020 [cited by examiner]
Tleis et al. “Contour expansion algorithm preprocessed by hough transform and circular shortest path for ovoid objects.” 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA). IEEE, 201… [cited by examiner]
Denk et al. (“Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure.” PLoS biology 2.11 (2004): e329.) (Year: 2004). [cited by examiner]
Tleis et al. “Contour expansion algorithm preprocessed by hough transform and circular shortest path for ovoid objects.” 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA). IEEE, 201… [cited by examiner]
Guray Akturk: “Multiplexed Immunohistochemical Consecutive Staining on Single Slide (MICSSS): Multiplexed Chromogenic IHC Assay for High-Dimensional Tissue Analysis”, Biomarkers for Immunotherapy of Cancer, Sep. 10, 201… [cited by applicant]
Ortiz De Solorzano C et al., “Segmentation of nuclei and cells using membrane related protein markers”, Journal of Microscopy, Blackwell Science, GB, vol. 201, No. 3, Mar. 1, 2001 (Mar. 1, 2001) , pp. 404-415. [cited by applicant]
Ali Can et al., “Techniques for Cellular Quantitation of Cancer Biomarkers” In: “Analytical Chemistry”, Jun. 1, 2018 (Jun. 1, 2018), American Chemical Society, vol. 89, pp. 1-29. [cited by applicant]
Extended European Search Report issued in corresponding European Patent Application No. 21189776.4-1210, dated Jan. 26, 2022. [cited by applicant]
Notice of Reasons for Refusal received in corresponding Japanese Patent Application No. 2020-141765, dated Apr. 2, 2024. [cited by applicant]