IP Library › Granted Patent US 11,100,359
Granted Patent B2
US 11,100,359 · App. 16/694,826 · Granted Aug 24, 2021

Method and system for analyzing image

Inventor: Minje Jang (Seoul, KR)
Assignee: Lunit Inc.
G06K9/469G06K9/00127G06K9/6256G06K9/6288G06T7/0012G06K2209/05G06T2207/20072G06T2207/20081G06T2207/20084G06T2207/30024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,100,359
App. No.
16/694,826
Filed
Nov 25, 2019
Granted
Aug 24, 2021
Kind
B2
Art Unit
2663
USPC
382/128
Abstract

An image analysis method and an image analysis system are disclosed. The method may include extracting training raw graphic data including at least one first node corresponding to a plurality of histological features of a training tissue slide image, and at least one first edge defined by a relationship between the histological features and generating training graphic data by sampling the first node of the training raw graphic data. The method may also include determining a parameter of a readout function by training a graph neural network (GNN) using the training graphic data and training output data corresponding to the training graphic data, and extracting inference graphic data including at least one second node corresponding to a plurality of histological features of an inference tissue slide image, and at least one second edge decided by a relationship between the histological features of the inference tissue slide image.

Claims (61)

1. An image analysis method comprising:

extracting training raw graphic data that includes at least one first node corresponding to a plurality of histological features of a training tissue slide image, and at least one first edge defined by a relationship between the histological features of the training tissue slide image;

generating training graphic data by sampling the first node of the training raw graphic data;

determining a parameter of a readout function by training a graph neural network (GNN) using the training graphic data and training output data corresponding to the training graphic data;

extracting inference graphic data that includes at least one second node corresponding to a plurality of histological features of an inference tissue slide image, and at least one second edge decided by a relationship between the histological features of the inference tissue slide image; and

deriving inference output data by the readout function after inputting the inference graphic data to the GNN.

2. The image analysis method according to claim 1 , wherein:

the first node is configured such that first feature information of the histological features of the training tissue slide image is embedded therein;

the first edge is configured such that second feature information about a relationship between the histological features of the training tissue slide image is embedded therein;

the second node is configured such that third feature information of the histological features of the inference tissue slide image is embedded therein; and

the second edge is configured such that fourth feature information of the histological features of the training tissue slide image is embedded therein.

3. The image analysis method according to claim 2 , wherein the determining includes:

determining a parameter of the readout function using the first feature information of the first node.

4. The image analysis method according to claim 2 , further comprising:

after the extracting, determining a parameter of an update function by training the GNN using both first feature information of a first target node and a first contiguous node from among the second nodes and second feature information of a first coupling edge connected to the first target node from among the first edges.

5. The image analysis method according to claim 1 , wherein the sampling of the first node is randomly performed.

6. The image analysis method according to claim 1 , wherein the extracting includes:

generating inference raw graphic data including the second node and the second edge; and

generating the inference graphic data by sampling the second node of the inference raw graphic data.

7. The image analysis method according to claim 6 , wherein the sampling of the second node is randomly performed.

8. The image analysis method according to claim 6 , wherein:

generating the inference graphic data includes generating N inference graphic data by sampling the second node N times; and

the deriving includes:

deriving each of N pre-inference output data about the N inference graphic data, and

generating the inference output data by merging the N pre-inference output data.

9. The image analysis method according to claim 8 , wherein generating the inference output data through merging includes:

generating the inference output data by averaging the N pre-inference output data.

10. The image analysis method according to claim 8 , wherein generating the inference output data through merging includes:

generating the inference output data by inputting the N pre-inference output data to a merge function,

wherein the merge function trains the GNN using the training graphic data and training output data corresponding to the training graphic data such that a parameter of the merge function is determined.

11. A non-transitory computer-readable recording medium storing computer-executable instructions, the computer-executable instructions configured to cause a computing device having at least one processor and at least one memory to perform an image analysis method comprising:

extracting training raw graphic data that includes at least one first node corresponding to a plurality of histological features of a training tissue slide image, and at least one first edge defined by a relationship between the histological features of the training tissue slide image;

generating training graphic data by sampling the first node of the training raw graphic data;

determining a parameter of a readout function and a parameter of an update function by training a graph neural network (GNN) using the training graphic data and training output data corresponding to the training graphic data;

extracting inference raw graphic data that includes at least one second node corresponding to a plurality of histological features of an inference tissue slide image, and at least one second edge decided by a relationship between the histological features of the inference tissue slide image;

generating training graphic data by sampling the second node of the inference raw graphic data;

updating feature information of the second node using the update function; and

deriving inference output data by the readout function after inputting the inference graphic data to the GNN.

12. The computer-readable recording medium according to claim 11 , wherein:

generating the inference graphic data by sampling the second node includes generating N inference graphic data by sampling the second node N times; and

the deriving includes:

deriving each of N pre-inference output data about the N inference graphic data, and

generating the inference output data by merging the N pre-inference output data.

13. An image analysis system comprising:

a storage unit configured to store a computer program therein; and

a processing unit configured to execute the computer program,

wherein the computer program includes executable-instructions configured to cause the processing unit to:

generate training raw graphic data that includes at least one first node corresponding to a plurality of histological features of a training tissue slide image, and at least one first edge defined by a relationship between the histological features of the training tissue slide image;

generate training graphic data by sampling the first node of the training raw graphic data;

determine a parameter of a readout function by training a graph neural network (GNN) using the training graphic data and training output data corresponding to the training graphic data;

extract inference raw graphic data that includes at least one second node corresponding to a plurality of histological features of an inference tissue slide image, and at least one second edge decided by a relationship between the histological features of the inference tissue slide image;

generate training graphic data by sampling the second node of the inference raw graphic data;

update feature information of the second node using the update function; and

derive inference output data by the readout function after inputting the inference graphic data to the graph neural network (GNN).

14. The image analysis system according to claim 13 , further comprising:

a pathology analysis device configured to extract training data from the training tissue slide image, and extract inference data from the inference tissue slide image,

wherein, in generating the training graphic data, the processing unit is configured to generate the training raw graphic data from the training data and generate the inference raw graphic data from the inference data.

15. The image analysis system according to claim 13 , wherein, in generating the inference graphic data by sampling the second node, the processing unit is configured to generate N inference graphic data by sampling the second node N times, and

wherein, in deriving the inference output data, the processing unit is configured to:

derive each of N pre-inference output data about the N inference graphic data, and

generate the inference output data by merging the N pre-inference output data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 25, 2019
From: JANG, MINJE
To: LUNIT INC.
Reel/Frame 051110/0345 →
Priority Claims (1)
KR 10-2019-0123098 · Oct 4, 2019 · national
Continuity (1)
Related Publication 20210103757A1 · Apr 8, 2021
Cited By (2)
US 12,475,564 US 12,664,650