IP Library Granted Patent US 11,042,778
Granted Patent B2
US 11,042,778 · App. 16/205,030 · Granted Jun 22, 2021

Generating realistic organ x-ray angiography (XA) images for deep learning consumption

Inventor: Omid Bonakdar Sakhi (North York, CA)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06K9/6255G06K9/6256G06N3/04G06N3/088G06T7/0014G16H30/20G06T2207/10116G06T2207/20081G06T2207/30048
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,042,778
App. No.
16/205,030
Granted
Jun 22, 2021
Kind
B2
Abstract

A method, computer system, and a computer program product for generating one or more realistic x-ray angiography (XA) images for deep learning is provided. The present invention may include generating at least one three-dimensional (3D) model associated with a subject. The present invention may then include projecting the generated 3D model associated with the subject into one or more two-dimensional (2D) images associated with the subject. The present invention may also include creating the one or more realistic XA images of the subject from the one or more 2D images associated with the subject by utilizing a trained deep convolutional neural network.

Claims (59)

1. A method for generating one or more realistic x-ray angiography (XA) images for deep learning, the method comprising:

generating two or more three-dimensional (3D) models associated with a subject, each of the models simulating a variation to an artery tree;

projecting each of the generated two or more 3D models associated with the subject into one or more two-dimensional (2D) images associated with the subject;

creating the one or more realistic XA images of the subject from the one or more 2D images associated with the subject by utilizing a trained deep convolutional neural network;

detecting, by a three-dimensional (3D) heart modeler, at least one disease associated with the subject; and

creating, automatically, a disease profile for the detected at least one disease associated with the subject.

2. The method of claim 1 , wherein each of the two or more simulated variations to an artery tree corresponds with a camera and an x-ray, and wherein one or more angles of the subject are stored in a database as a part of the ground-truth labeling.

3. The method of claim 1 in which the subject includes an organ associated with a living organism.

4. The method of claim 2 , further comprising:

analyzing the two or more simulated variations to an artery tree;

generating the one or more 2D images associated with the subject for each of the analyzed two or more simulated variations to an artery tree; and

modifying a plurality of image quality settings associated with the generated one or more 2D images.

5. The method of claim 1 , wherein generating two or more three-dimensional (3D) models associated with a subject, each of the models simulating a variation to an artery tree, further comprises:

retrieving, automatically, at least one pre-existing model associated with the subject from an external source; and

transmitting the retrieved at least one pre-existing model associated with the subject to a computing device.

6. The method of claim 1 , further comprising:

gathering the generated one or more 2D images associated with a target domain and one or more synthetic images associated with a source domain;

adding, by utilizing a Cycle GAN model, at least one form of relevant background to each of the one or more synthetic images; and

building the trained deep convolutional neural network to map one or more synthetic images from the source domain into one or more realistic XA images from the target domain.

7. A computer system for generating one or more realistic x-ray angiography (XA) images for deep learning, comprising:

one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:

generating two or more three-dimensional (3D) models associated with a subject, each of the models simulating a variation to an artery tree;

projecting each of the generated two or more 3D models associated with the subject into one or more two-dimensional (2D) images associated with the subject;

creating the one or more realistic XA images of the subject from the one or more 2D images associated with the subject by utilizing a trained deep convolutional neural network;

detecting, by a three-dimensional (3D) heart modeler, at least one disease associated with the subject; and

creating, automatically, a disease profile for the detected at least one disease associated with the subject.

8. The computer system of claim 7 , wherein each of the two or more simulated variations to an artery tree corresponds with a camera and an x-ray, and wherein one or more angles of the subject are stored in a database as a part of the ground-truth labeling.

9. The computer system of claim 7 in which the subject includes an organ associated with a living organism.

10. The computer system of claim 8 , further comprising:

analyzing the two or more simulated variations to an artery tree;

generating the one or more 2D images associated with the subject for each of the analyzed two or more simulated variations to an artery tree; and

modifying a plurality of image quality settings associated with the generated one or more 2D images.

11. The computer system of claim 7 , wherein generating two or more three-dimensional (3D) models associated with a subject, each of the models simulating a variation to an artery tree, further comprises:

retrieving, automatically, at least one pre-existing model associated with the subject from an external source; and

transmitting the retrieved at least one pre-existing model associated with the subject to a computing device.

12. The computer system of claim 7 , further comprising:

gathering the generated one or more 2D images associated with a target domain and one or more synthetic images associated with a source domain;

adding, by utilizing a Cycle GAN model, at least one form of relevant background to each of the one or more synthetic images; and

building the trained deep convolutional neural network to map one or more synthetic images from the source domain into one or more realistic XA images from the target domain.

13. A computer program product for generating one or more realistic x-ray angiography (XA) images for deep learning, comprising:

one or more computer-readable storage media and program instructions stored on at least one of the one or more tangible storage media, the program instructions executable by a processor to cause the processor to perform a method comprising:

generating two or more three-dimensional (3D) models associated with a subject, each of the models simulating a variation to an artery tree;

projecting each of the generated two or more 3D models associated with the subject into one or more two-dimensional (2D) images associated with the subject;

creating the one or more realistic XA images of the subject from the one or more 2D images associated with the subject by utilizing a trained deep convolutional neural network;

detecting, by a three-dimensional (3D) heart modeler, at least one disease associated with the subject; and

creating, automatically, a disease profile for the detected at least one disease associated with the subject.

14. The computer program product of claim 13 , wherein each of the two or more simulated variations to an artery tree corresponds with a camera and an x-ray, and wherein one or more angles of the subject are stored in a database as a part of the ground-truth labeling.

15. The computer program product of claim 13 in which the subject includes an organ associated with a living organism.

16. The computer program product of claim 14 , further comprising:

analyzing the two or more simulated variations to an artery tree;

generating the one or more 2D images associated with the subject for each of the analyzed two or more simulated variations to an artery tree; and

modifying a plurality of image quality settings associated with the generated one or more 2D images.

17. The computer program product of claim 13 , wherein generating two or more three-dimensional (3D) models associated with a subject, each of the models simulating a variation to an artery tree, further comprises:

retrieving, automatically, at least one pre-existing model associated with the subject from an external source; and

transmitting the retrieved at least one pre-existing model associated with the subject to a computing device.

18. The computer program product of claim 13 , further comprising:

gathering the generated one or more 2D images associated with a target domain and one or more synthetic images associated with a source domain;

adding, by utilizing a Cycle GAN model, at least one form of relevant background to each of the one or more synthetic images; and

building the trained deep convolutional neural network to map one or more synthetic images from the source domain into one or more realistic XA images from the target domain.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2022
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: MERATIVE US L.P.
Reel/Frame 061496/0752 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 29, 2018
From: BONAKDAR SAKHI, OMID
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 047629/0268 →