IP Library Granted Patent US 11,972,575
Granted Patent B2
US 11,972,575 · App. 16/892,311 · Granted Apr 30, 2024

Systems and methods for generating augmented segmented image set

Inventors: Supratik Bose (Walnut Creek, CA); Jonathan Maltz (Walnut Creek, CA)
Assignee: UIH AMERICA, INC.
G06T7/168A61N5/1039G06T7/11G06T2207/10081G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,972,575
App. No.
16/892,311
Granted
Apr 30, 2024
Kind
B2
Abstract

A system and method for generating augmented segmented image set obtain are provided. The method may include: obtaining a first image including a first anatomical structure of a first object; determining first feature data of the first anatomical structure; determining one or more first transformations related to the first anatomical structure, wherein a first transformation includes a transformation type and one or more transformation parameters related to the transformation type; applying the one or more first transformations to the first feature data of the first anatomical structure to generate second feature data of the first anatomical structure; and generating a second image based on the second feature data of the first anatomical structure.

Claims (70)

1. A method implemented on a computing device having at least one processor, and at least one computer-readable storage medium, the method comprising:

obtaining a segmentation model configured to identify a plurality of pixels representing an anatomical structure in an image, wherein the segmentation model is determined by training an initial model using one or more second images, and the one or more second images are artificial images generated based on one or more first images, each of the one or more second images is generated by a process including:

extracting a first anatomical structure from one of the one or more first images;

generating a transformed anatomical structure by transforming at least one of an area, a location, or a shape of a contour of the extracted first anatomical structure; and

generating the second images based on the transformed anatomical structure;

obtaining a treatment image relating to the anatomical structure; and

identifying a plurality of pixels representing the anatomical structure in the treatment image using the segmentation model.

2. The method of claim 1 , wherein the identifying a plurality of pixels representing the anatomical structure in the treatment image includes:

identifying pixel values or pixel locations of the plurality of pixels.

3. The method of claim 1 , further comprising:

determining segment information associated with the anatomical structure by segmenting, using the segmentation model, the treatment image.

4. The method of claim 3 , wherein the segment information associated with the anatomical structure includes at least one of a contour of the anatomical structure or an area of the anatomical structure.

5. The method of claim 1 , wherein the one or more second images are generated by the process comprising:

obtaining first feature data of the first anatomical structure in the one or more first images;

determining one or more transformations related to the first anatomical structure in the one or more first images;

generating second feature data related to the transformed anatomical structure by applying the one or more transformations to the first feature data; and

generating the one or more second images based on the second feature data.

6. The method of claim 5 , wherein the one or more transformations include at least one of a transformation type or one or more transformation parameters related to the transformation type.

7. The method of claim 6 , wherein the transformation type includes at least one of a displacement of the first anatomical structure, a rotation of the first anatomical structure, a deformation of the first anatomical structure, an appearance of an extra structure, or a disappearance of partial of the first anatomical structure.

8. The method of claim 7 , wherein the one or more transformation parameters related to the transformation type include at least one of a displacement distance, a displacement direction, a rotation angle, a deformation direction, or a deformation scale.

9. The method of claim 5 , wherein the segmentation model is determined by a process comprising:

for each of at least one of the one or more second images,

inputting the second image into the initial model;

outputting feature data of the anatomical structure in the second image;

generating an error function based on a difference between the outputted feature data of the anatomical structure and the second feature data of the anatomical structure in the second image; and

updating one or more parameters of the initial model based on the error function; and

determining the segmentation model based on the one or more updated parameters.

10. The method of claim 9 , wherein the one or more parameters include weights of kernels of the initial model.

11. The method of claim 1 , wherein the segmentation model includes at least one of a convolutional neural network, an adaptive boosting model, or a gradient boosting decision tree model.

12. The method of claim 5 , wherein the generating the second feature data related to the transformed anatomical structure by applying the one or more transformations to the first feature data includes:

determining a reference frame by determining one or more fixed points in the first image; and

determining the second feature data related to the transformed anatomical structure based on the reference frame and the one or more transformation parameters.

13. The method of claim 5 , wherein the generating the second feature data related to the transformed anatomical structure by applying the one or more transformations to the first feature data includes:

determining a center point of the first anatomical structure;

determining a displacement vector based on the one or more transformation parameters; and

determining the second feature data based on the displacement vector and the center point of the first anatomical structure.

14. The method of claim 1 , wherein the segmentation model is determined by training the initial model using a combination of the one or more first images and the one or more second images.

15. A system comprising:

a non-transitory storage medium storing a set of instructions;

a processor in communication with the non-transitory storage medium, wherein when executing the set of instructions, the processor is directed to perform operations including:

obtaining a segmentation model configured to identify a plurality of pixels representing an anatomical structure in an image, wherein the segmentation model is determined by training an initial model using one or more second images, and the one or more second images are artificial images generated based on one or more first images, each of the one or more second images is generated by a process including:

extracting a first anatomical structure from one of the one or more first images;

generating a transformed anatomical structure by transforming at least one of an area, a location, or a shape of a contour of the extracted first anatomical structure; and

generating the second image based on the transformed anatomical structure;

obtaining a treatment image relating to the anatomical structure; and

identifying a plurality of pixels representing the anatomical structure in the treatment image using the segmentation model.

16. The system of claim 15 , wherein to identify a plurality of pixels representing the anatomical structure in the treatment image, the processor is directed to perform the operations including:

identifying pixel values or pixel locations of the plurality of pixels.

17. The system of claim 15 , wherein the processor is directed further to perform the operations including:

determining segment information associated with the anatomical structure by segmenting, using the segmentation model, the treatment image.

18. The system of claim 15 , wherein the one or more second images are generated by the process comprising:

obtaining first feature data of the first anatomical structure in the one or more first images;

determining one or more transformations related to the first anatomical structure in the one or more first images;

generating second feature data related to the transformed anatomical structure by applying the one or more transformations to the first feature data; and

generating the one or more second images based on the second feature data.

19. The system of claim 18 , wherein the segmentation model is determined by a process comprising:

for each of at least one of the one or more second images,

inputting the second image into the initial model;

outputting feature data of the anatomical structure in the second image;

generating an error function based on a difference between the outputted feature data of the anatomical structure and the second feature data of the anatomical structure in the second image;

updating one or more parameters of the initial model based on the error function; and

determining the segmentation model based on the one or more updated parameters.

20. A non-transitory computer readable medium, comprising:

instructions being executed by at least one processor, causing the at least one processor to implement a method, the method comprising:

obtaining a segmentation model configured to identify a plurality of pixels representing an anatomical structure in an image, wherein the segmentation model is determined by training an initial model using one or more second images, and the one or more second images are artificial images generated based on one or more first images, each of the one or more second images is generated by a process including:

extracting a first anatomical structure from one of the one or more first images;

generating a transformed anatomical structure by transforming at least one of an area, a location, or a shape of a contour of the extracted first anatomical structure; and

generating the second images based on the transformed anatomical structure;

obtaining a treatment image relating to the anatomical structure; and

identifying a plurality of pixels representing the anatomical structure in the treatment image using the segmentation model.

Assignments (3)
CHANGE OF NAME Recorded Jan 15, 2025
From: UIH AMERICA, INC.
To: UNITED IMAGING HEALTHCARE NORTH AMERICA, INC.
Reel/Frame 069899/0414 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 19, 2022
From: UIH-RT US LLC
To: UIH AMERICA, INC.
Reel/Frame 062136/0158 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2020
From: BOSE, SUPRATIK; MALTZ, JONATHAN
To: UIH-RT US LLC
Reel/Frame 052845/0191 →
Continuity (2)
Continuation 16004387 · Jun 9, 2018
Related Publication 20200302613A1 · Sep 24, 2020