IP Library › Granted Patent US 12,322,112
Granted Patent B2
US 12,322,112 · App. 17/683,643 · Granted Jun 3, 2025

Methods and apparatuses for visualization of tumor segmentation

Inventors: Michal Holtzman Gazit (Haifa, IL); Reuven Ruby Shamir (Haifa, IL)
Assignee: Novocure GmbH
G06T7/11G06T7/136G06T2207/20081G06T2207/30096
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,322,112
App. No.
17/683,643
Granted
Jun 3, 2025
Kind
B2
Abstract

A computer-implemented method for tumor segmentation, the method comprises obtaining image data of a region of interest of a subject's body, wherein the region of interest corresponds to a tumor of the subject's body, generating two or more tumor segmentation predictions based on the image data, calculating a divergence between the two or more tumor segmentation predictions, and generating a visualization of tumor segmentation uncertainty based on the calculated divergence between the two or more tumor segmentation predictions.

Claims (43)

1. A computer-implemented method for tumor segmentation, the method comprises:

obtaining image data of a region of interest of a subject's body, wherein the region of interest corresponds to a tumor of the subject's body;

generating two or more tumor segmentation predictions based on the image data using two or more trained segmentation networks;

calculating a divergence between the two or more tumor segmentation predictions; and

generating a visualization of tumor segmentation uncertainty based on the calculated divergence between the two or more tumor segmentation predictions.

2. The method of claim 1 , further comprising:

obtaining anatomical information and surgical information of the subject's body, wherein the two or more tumor segmentation predictions are generated based on the image data, the anatomical information, and the surgical information.

3. The method of claim 2 , further comprising:

adding or removing anatomical structures from the visualization based on the anatomical information and surgical information of the subject's body.

4. The method of claim 1 , wherein the divergence is calculated through a symmetric Kullback-Leibler (KL) divergence loss.

5. The method of claim 1 , wherein the two or more tumor segmentation predictions are generated by segmentation networks, wherein each of the segmentation networks comprises a variational auto-encoder that reconstructs the image data from a shared encoder parameter.

6. The method of claim 1 , wherein the visualization comprises an image of the subject with a segmentation prediction and an uncertainty map, wherein the uncertainty map is based on the calculated divergence between the two or more tumor segmentation predictions.

7. The method of claim 6 , further comprising:

removing voxels from the segmentation prediction having an uncertainty greater than a threshold.

8. A computer-implemented method for tumor segmentation, the method comprises

obtaining two or more segmentation networks trained using a common training set with a plurality of training losses, the common training set comprising images of other subjects;

obtaining image data of a region of interest of a subject's body, wherein the region of interest corresponds to a tumor of the subject's body;

generating two or more tumor segmentation predictions based on the image data and the two or more segmentation networks;

calculating a divergence between the two or more tumor segmentation predictions; and

generating a visualization of tumor segmentation uncertainty based on the calculated divergence between the two or more tumor segmentation predictions.

9. The method of claim 8 , wherein the segmentations networks are trained in parallel using a unified loss, wherein the unified loss comprises at least one of a segmentation loss or a divergence loss.

10. The method of claim 8 , further comprising:

applying augmentation to the training data set to obtain an augmented training set, wherein the augmentation comprises at least one of: intensity normalization, random shift, random scale-up of intensity value, random flip, or random scaling of an image patch; and

training the segmentation networks using the augmented training data set.

11. The method of claim 8 , further comprising:

obtaining anatomical information and surgical information of the subject's body, wherein the two or more tumor segmentation predictions are generated based on the image data, the two or more trained segmentation networks, the anatomical information, and the surgical information.

12. The method of claim 11 , further comprising:

adding or removing anatomical structures from the visualization based on the anatomical information and surgical information of the subject's body.

13. The method of claim 8 , further comprising:

determining locations on the subject's body to place transducers for applying tumor treating fields based on at least one of the visualization of tumor segmentation uncertainty or two or more tumor segmentation predictions.

14. An apparatus comprising: one or more processors; and memory storing processor-executable instructions that, when executed by the one or more processors, cause the apparatus to:

obtain a training data set of images of other subjects;

train two or more segmentation networks using the training set and a unified loss based on a comparison between training outputs of the two or more segmentation networks; and

obtain image data of a region of interest of a subject's body, wherein the region of interest corresponds to a tumor of the subject's body;

generate two or more segmentation predictions based on the image data and the two or more trained segmentation networks; and

calculate a divergence between the two or more segmentation predictions; and

generate a visualization of tumor segmentation uncertainty based on the segmentation prediction, wherein the visualization of segmentation uncertainty is generated based on a calculated divergence between the two or more segmentation predictions.

15. The apparatus of claim 14 , wherein the unified loss comprises a segmentation loss, a divergence loss, and a reconstruction loss.

16. The apparatus of claim 15 , wherein the segmentation loss comprises a Dice coefficient and a cross entropy loss.

17. The apparatus of claim 15 , wherein the divergence loss comprises a Kullback-Leibler divergence loss.

18. The apparatus of claim 15 , wherein each of the segmentation networks comprises a variational auto-encoder, and the reconstruction loss comprises a reconstruction loss of the variational auto-encoder for each segmentation network.

19. The apparatus of claim 15 , wherein the unified loss comprises a parameter to balance the segmentation loss and the divergence loss.

20. The apparatus of claim 14 , wherein the memory storing processor-executable instructions that, when executed by the one or more processors, further cause the apparatus to: obtain anatomical information and surgical information of the subject's body, wherein the two or more segmentation predictions are generated based on the image data, the two or more trained segmentation networks, the anatomical information, and the surgical information.

Assignments (2)
PATENT SECURITY AGREEMENT Recorded May 4, 2024
From: NOVOCURE GMBH (SWITZERLAND)
To: BIOPHARMA CREDIT PLC
Reel/Frame 067315/0399 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 22, 2022
From: HOLTZMAN GAZIT, MICHAL; SHAMIR, REUVEN RUBY
To: NOVOCURE GMBH
Reel/Frame 060271/0292 →
Continuity (3)
Provisional Application 63155564 · Mar 2, 2021
Provisional Application 63155626 · Mar 2, 2021
Related Publication 20220284585A1 · Sep 8, 2022
References Cited (36)
US 7565205B2 · Palti · 2009 [cited by applicant]
US 11464491B2 · Jacob · 2022 [cited by examiner]
US 20100322489A1 · Tizhoosh · 2010 [cited by examiner]
US 20150327766A1 · Gupta · 2015 [cited by examiner]
US 20190026897A1 · Wu · 2019 [cited by examiner]
US 20200167930A1 · Wang · 2020 [cited by examiner]
US 20200219261A1 · Shamir · 2020 [cited by examiner]
US 20220036135A1 · Hu et al. · 2022 [cited by applicant]
CN 110517759A · 2019 [cited by applicant]
Spyridon Bakas, et al., “Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the Brats challenge”, (2019), 49 pages. [cited by applicant]
Matthew T. Ballo, et al., “Correlation of Tumor Treating Fields Dosimetry to Survival Outcomes in Newly Diagnosed Glioblastoma: A Large-Scale Numerical Simulation-Based Analysis of Data from the Phase 3 EF-14 Randomized… [cited by applicant]
Ken Chang, et al., “Automatic assessment of glioma burden: a deep learning algorithm for fully automated volumetric and bidimensional measurement,” Neuro-Oncology, 21(11), (2019), pp. 1412-1422. [cited by applicant]
Filipe R. Cordeiro, et al., “A survey on deep learning with noisy labels: How to train your model when you cannot trust on the annotations?”, 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), (2020),… [cited by applicant]
James S. Cordova, et al., “Quantitative tumor segmentation for evaluation of extent of glioblastoma resection to facilitate multisite clinical trials”, Translational Oncology, vol. 7, No. 1, (Feb. 2014), pp. 40-47. [cited by applicant]
Ekin Ermis et al., “Fully automated brain resection cavity delineation for radiation target volume definition in glioblastoma patients using deep learning”, Radiation Oncology, 15:100 (May 2020), 10 pages. [cited by applicant]
Bo Han et al., “Co-teaching: Robust training of deep neural networks with extremely noisy labels”, In: NeurIPS, (2018), 13 pages. [cited by applicant]
Christian Henker, et al., “Volumetric quantification of glioblastoma: experiences with different measurement techniques and impact on survival,” J. Neurooncol , 135, (2017), pp. 391-402. [cited by applicant]
Davood Karimi et al., “Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis”, Medical Image Analysis 65, 101759 (2020), 49 pages. [cited by applicant]
Diederik P. Kingma et al., “Adam: A method for stochastic optimization,” ICLR 2015, (2015), pp. 1-15. [cited by applicant]
Eilon D. Kirson, et al., “Alternating electric fields arrest cell proliferation in animal tumor models and human brain tumors”, PNAS, vol. 104, No. 24, (2007), pp. 10152-10157. [cited by applicant]
Sangjune Lee, et al., “Improved dosimetric accuracy with semi-automatic contour propagation of organs-at-risk in glioblastoma patients undergoing chemoradiation,” Journal of Applied Clinical Medical Physics 20:12, (2019… [cited by applicant]
Raghav Mehta, et al., “Uncertainty evaluation metric for brain tumour segmentation,” Medical Imaging with Deep Learning, (2020), 8 pages. [cited by applicant]
Raphael Meier et al., “Patient-specific semi-supervised learning for postoperative brain tumor segmentation,” Medical Image Computing and Computer-Assisted Intervention—MICCAI 2014. vol. 17, (2014), pp. 714-721. [cited by applicant]
Bjoern H. Menze, et al., “The multimodal brain tumor image segmentation benchmark (Brats)”, IEEE Transactions on Medical Imaging vol. 34, No. 10, (2015), pp. 1993-2024. [cited by applicant]
Andriy Myronenko, “3D MRI brain tumor segmentation using autoencoder regularization,” (2019), 10 pages. [cited by applicant]
Tanya Nair et al., “Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation”, Medical Image Computing and Computer Assisted Intervention—MICCAI 2018, Springer Internation… [cited by applicant]
Adam Paszke et al., “Pytorch: An imperative style, high-performance deep learning library,” 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), (2019), 12 pages. [cited by applicant]
Mengye Ren et al., “Learning to reweight examples for robust deep learning,” 35th ICML (2018), 13 pages. [cited by applicant]
Olaf Ronneberger, et al., “U-net: Convolutional networks for biomedical image segmentation,” Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, (2015), 8 pages. [cited by applicant]
Hongxin Wei, et al., “Combating noisy labels by agreement: A joint training method with co-regularization,” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2020), 11 pages. [cited by applicant]
Steven V. Weijs et al., “Kullback-Leibler divergence as a forecast skill score with classic reliability-resolution-uncertainty decomposition”, American Meterorological Society, Monthly Weather Review, vol. 138, (2010), … [cited by applicant]
Guotai Wang et al., “Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks,” Feb. 1, 2019, 13 pages. [cited by applicant]
Alain Jungo et al., “Towards Uncertainty-Assisted Brain Tumor Segmentation and Survival Prediction,” 2018, pp. 474-485. [cited by applicant]
Alex Kendall et al., “Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding,” Oct. 10, 2016, 11 pages. [cited by applicant]
Tanya Nair et al., “Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation,” Oct. 16, 2018, 15 pages. [cited by applicant]
Moloud Abdar et al., “A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges,” Jan. 6, 2021, 68 pages. [cited by applicant]