IP Library › Granted Patent US 12,614,281
Granted Patent B2
US 12,614,281 · App. 17/658,924 · Granted Apr 28, 2026

Image analysis system for identifying lung features

Inventors: Kenneth Alan Koster (San Francisco, CA); Russell Kenji Yoshinaka (Lafayette, CA); Hanna Katherine Winter (San Francisco, CA)
Assignee: Ceevra, Inc.
G06T7/11A61B5/004A61B5/489A61B6/032A61B6/50G06T7/13G06T7/62G06T7/70G06V10/26G06V10/44G06V20/64G06T2200/04G06T2207/10081G06T2207/10088G06T2207/20128G06T2207/30061G06T2207/30096G06T2207/30101G06V2201/031
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,614,281
App. No.
17/658,924
Granted
Apr 28, 2026
Kind
B2
Abstract

Methods and apparatuses for identifying lung features are provided herein.

Claims (54)

1 . A method of determining segment boundaries of lung segments, the method comprising:

receiving image data forming a three-dimensional representation of at least a part of a lung, wherein the lung is divided into multiple lung lobes, wherein each lobe is divided into multiple lung segments;

computationally identifying, using the image data, at least one anatomical feature within the lung, wherein the at least one anatomical feature comprises one or more of the following: a network of arteries, a network of veins, a network of airways, some or all of lung parenchyma, and one or more fissures of the lung;

computationally identifying, using the at least one anatomical feature, all segment boundaries of at least one lung segment within the lung, wherein the segment boundaries comprise surfaces between the at least one lung segment and adjacent lung segments; and

generating a representation of all or part of the lung showing the segment boundaries of the at least one lung segment within the lung.

2 . The method of claim 1 , wherein the image data comprises a one or both of a CT scan and an MRI.

3 . The method of claim 1 , wherein the at least one anatomical feature comprises some or all of the lung parenchyma and the network of arteries.

4 . The method of claim 1 , wherein the at least one anatomical feature comprises some or all of the lung parenchyma and the network of veins.

5 . The method of claim 1 , wherein the at least one anatomical feature comprises some or all of the lung parenchyma and the network of airways.

6 . The method of claim 1 , wherein the at least one anatomical feature comprises the one or more fissures of the lung.

7 . The method of claim 1 , wherein computationally identifying all segment boundaries of the at least one lung segment comprises:

computationally identifying, using the at least one anatomical feature, one or more of the following: segmental branches of arteries and related subtrees, segmental branches of veins, segmental branches of airways and related subtrees, and the fissures of the lung;

generating segment boundaries between adjacent lung segments in the lung through analysis of segmental branches of arteries and related subtrees, segmental branches of veins, segmental branches of airways and related subtrees, and/or fissures of the lung; and

combining the segment boundaries between adjacent lung segments to define all of the segment boundaries of the at least one lung segment within the lung.

8 . The method of claim 1 , further comprising:

computationally identifying, using the at least one anatomical feature, lobe boundaries for a plurality of lobes in the lung.

9 . The method of claim 1 , further comprising:

computationally identifying, using the image data, a lesion within the lung; and

computationally determining, from the image data, that the lesion is located in a given lung segment of the at least one lung segment or in a region spanning two or more lung segments.

10 . The method of claim 9 , further comprising:

computationally measuring, from the image data, a minimum distance between the lesion and a segment boundary of any of the lung segments.

11 . A method of identifying segment boundaries of lung segments, the method comprising:

receiving image data forming a volumetric representation of at least a part of a lung, wherein the lung is divided into multiple lobes, wherein each lobe is divided into multiple lung segments;

computationally identifying, using the image data, a network of arteries;

computationally identifying, using the image data, a network of bronchi;

computationally identifying, using the image data, a network of veins; and

computationally identifying, based on the identified network of arteries, network of bronchi, and network of veins, segment boundaries of a plurality of lung segments within at least one lobe of the lung, wherein the segment boundaries comprise surfaces between adjacent lung segments.

12 . The method of claim 11 , wherein the image data comprises one or both of a CT scan and an MRI.

13 . The method of claim 11 , wherein computationally identifying the network of arteries comprises:

computationally identifying a tube-like structure in the image data, wherein the tube-like structure is identified by identifying a set of gradient changes within the image data; and

computationally determining, using the image data and based on how the tube-like structure branches within the lung, that the tube-like structure is part of the network of arteries.

14 . The method of claim 11 , wherein computationally identifying the network of bronchi comprises:

computationally identifying a tube-like structure in the image data, wherein the additional tube-like structure is identified by identifying a set of gradient changes within the image data; and

computationally determining, using the image data and based on how the additional tube-like structure branches within the lung, that the tube-like structure is part of the network of bronchi.

15 . The method of claim 11 , wherein computationally identifying the network of veins comprises:

computationally identifying a tube-like structure in the image data, wherein the tube-like structure is identified by identifying a set of gradient changes within the image data; and

computationally determining, using the image data and based on how the tube-like structure branches within the lung, that the tube-like structure is part of the network of veins.

16 . The method of claim 11 , wherein computationally identifying the segment boundaries of the plurality of lung segments comprises:

computationally identifying a volume within the image data that receives blood from a section of the network of arteries.

17 . The method of claim 11 , wherein computationally identifying the segment boundaries of the plurality of lung segments comprises:

computationally identifying a volume within the image data that receives air from a section of the network of bronchi.

18 . The method of claim 11 , further comprising:

computationally identifying, using the image data, one or more intersegmental veins; and

computationally refining, based on the one or more intersegmental veins, the segment boundaries of the plurality of lung segments of the at least one lobe.

19 . A method of determining segment boundaries of lung segments, the method comprising:

receiving image data forming a three-dimensional representation of at least a part of a lung, wherein the lung is divided into multiple lobes, wherein each lobe is divided into multiple lung segments;

computationally identifying, using the image data, one or more intersegmental veins within the lung;

computationally identifying segment boundaries of at least one lung segment within the lung based at least in part on the identified one or more intersegmental veins, wherein the segment boundaries comprise surfaces between the at least one lung segment and adjacent lung segments; and

generating a representation of all or part of the lung showing the segment boundaries of the at least one lung segment within the lung.

20 . A method of determining segment boundaries of lung segments, the method comprising:

receiving image data forming a three-dimensional representation of at least a part of a lung, wherein the lung is divided into multiple lobes, wherein each lobe is divided into multiple lung segments;

computationally identifying, using the image data, one or more fissures of the lung;

computationally identifying segment boundaries of at least one lung segment within the lung based at least in part on the identified one or more fissures, wherein the segment boundaries comprise surfaces between the at least one lung segment and adjacent lung segments; and

generating a representation of all or part of the lung showing the segment boundaries of the at least one lung segment within the lung.

Continuity (3)
Continuation 16949685 · Nov 10, 2020
Provisional Application 62933884 · Nov 11, 2019
Related Publication 20220237805A1 · Jul 28, 2022
References Cited (97)
US 8150113B2 · Ray et al. · 2012 [cited by applicant]
US 8244733B2 · Fortier et al. · 2012 [cited by applicant]
US 8538770B2 · Papier et al. · 2013 [cited by applicant]
US 9129054B2 · Nawana et al. · 2015 [cited by applicant]
US 9538925B2 · Sharma et al. · 2017 [cited by applicant]
US 9797380B2 · Burkett · 2017 [cited by applicant]
US 10653502B2 · Kuo · 2020 [cited by applicant]
US 11348250B2 · Koster et al. · 2022 [cited by applicant]
US 20040122704A1 · Sabol et al. · 2004 [cited by applicant]
US 20050049497A1 · Krishnan et al. · 2005 [cited by applicant]
US 20080201280A1 · Martin et al. · 2008 [cited by applicant]
US 20080235052A1 · Node-Langlois et al. · 2008 [cited by applicant]
US 20080317314A1 · Schwartz et al. · 2008 [cited by applicant]
US 20090185731A1 · Ray et al. · 2009 [cited by applicant]
US 20090226057A1 · Mashiach et al. · 2009 [cited by applicant]
US 20090252395A1 · Chan et al. · 2009 [cited by applicant]
US 20090299766A1 · Friedlander et al. · 2009 [cited by applicant]
US 20100070293A1 · Brown et al. · 2010 [cited by applicant]
US 20100191071A1 · Anderson et al. · 2010 [cited by applicant]
US 20110022622A1 · Boroczky et al. · 2011 [cited by applicant]
US 20110119212A1 · De Bruin et al. · 2011 [cited by applicant]
US 20110313790A1 · Yao · 2011 [cited by applicant]
US 20120232930A1 · Schmidt et al. · 2012 [cited by applicant]
US 20120274631A1 · Friedland et al. · 2012 [cited by applicant]
US 20120283574A1 · Park et al. · 2012 [cited by applicant]
US 20130325508A1 · Johnson et al. · 2013 [cited by applicant]
US 20140079306A1 · Inoue · 2014 [cited by examiner]
US 20140298270A1 · Wiemker · 2014 [cited by examiner]
US 20150126894A1 · Benson et al. · 2015 [cited by applicant]
US 20160067007A1 · Piron et al. · 2016 [cited by applicant]
US 20160110632A1 · Kiraly et al. · 2016 [cited by applicant]
US 20160140300A1 · Purdie et al. · 2016 [cited by applicant]
US 20160328850A1 · Yin et al. · 2016 [cited by applicant]
US 20160338685A1 · Nawana et al. · 2016 [cited by applicant]
US 20170035514A1 · Fox et al. · 2017 [cited by applicant]
US 20170061375A1 · Laster et al. · 2017 [cited by applicant]
US 20170076046A1 · Barnes et al. · 2017 [cited by applicant]
US 20170193160A1 · Long et al. · 2017 [cited by applicant]
US 20170224301A1 · Radhakrishnan · 2017 [cited by examiner]
US 20180040088A1 · Kanada · 2018 [cited by applicant]
US 20180047168A1 · Chen · 2018 [cited by examiner]
US 20180153632A1 · Tokarchuk et al. · 2018 [cited by applicant]
US 20180303552A1 · Ryan et al. · 2018 [cited by applicant]
US 20180344308A1 · Nawana et al. · 2018 [cited by applicant]
US 20180368930A1 · Esterberg et al. · 2018 [cited by applicant]
US 20190139227A1 · Wang · 2019 [cited by examiner]
US 20190325645A1 · Guendel · 2019 [cited by examiner]
US 20190370964A1 · Yu et al. · 2019 [cited by applicant]
US 20200349703A1 · Hashimoto · 2020 [cited by examiner]
US 20200387729A1 · Ichinose · 2020 [cited by examiner]
US 20200410670A1 · Gerard et al. · 2020 [cited by applicant]
US 20210142485A1 · Koster et al. · 2021 [cited by applicant]
US 20210169576A1 · Koster et al. · 2021 [cited by applicant]
US 20220092791A1 · Dougherty · 2022 [cited by examiner]
WO WO2017189758A1 · 2017 [cited by applicant]
WO WO2017197360A1 · 2017 [cited by applicant]
WO WO2019184158A1 · 2019 [cited by applicant]
Aurenhammer, Franz, “Voronoi diagrams—A survey of a fundamental geometric data structure,” ACM Computing Surveys, vol. 23, No. 3, Sep. 1991, pp. 345-405. [cited by applicant]
Benmansour, Fethalah, et al. “Tubular Structure Segmentation Based on Minimal Path Method and Anisotropic Enhancement,” International Journal of Computer Vision 92.2, Mar. 31, 2010, pp. 192-210. [cited by applicant]
Benmansour, Fethallah, et al., “Tubular Geodesics Using Oriented Flux: An ITK Implementation,” Article, Feb. 1, 2013, 8 pages. [cited by applicant]
Beucher, Serge. “The Watershed Transformation Applied to Image Segmentation,” Scanning Microscopy—Supplement, 1992, pp. 1-26. [cited by applicant]
Cornea, Nicu D., et al. “Computing Hierarchical Curve-Skeletons of 3D Objects,” The Visual Computer 21.11, Oct. 2005, 19 pages. [cited by applicant]
European Office Action dated Apr. 12, 2022 in Application No. EP19846994.2. [cited by applicant]
Extended European Search Report dated Mar. 23, 2022, in Application No. 19846994.2. [cited by applicant]
FujiFilm, Synapse 3D, Product Data Sheet, Version 4, Enterprise Solution, http://www.fujifilmusa.com/products/medical/radiology/3D, MKT-0033341-B, 15 pages. [cited by applicant]
FujiFilm, Synapse 3D V4.1US, Synapse 3D (US) Product Specifications: Z30N1138B, Fujifilm Corporation, Sep. 2014, 62 pages. [cited by applicant]
Giuliani, Nicola, et al. “Pulmonary Lobe Segmentation in CT Images using Alpha-Expansion,” Visigrapp 2018, vol. 4, pp. 387-394. [cited by applicant]
Gu, Suicheng, et al. “Automated Lobe-Based Airway Labeling,” International Journal of Biomedical Imaging, 2012 (2012): 10 pages. [cited by applicant]
Helen, R., et al. “Segmentation of Pulmonary Parenchyma in CT Lung Images Based on 2D Otsu Optimized by PSO,” 2011 International Journal of Advanced Science and Technology, vol. 29 No. 10S, 2020, pp. 4334-4347. [cited by applicant]
Helmberger, Michael, et al. “Quantification of Tortuosity and Fractal Dimension of The Lung Vessels In Pulmonary Hypertension Patients,” PLOS | One, Jan. 14, 2014, vol. 9, Issue 1, 87515, pp. 1-9. [cited by applicant]
International Preliminary Report on Patentability dated May 27, 2022, in PCT Application No. PCT/US2020/059988. [cited by applicant]
International Preliminary Report on Patentability mailed Feb. 18, 2021, issued in PCT Application No. PCT/US2019/045129. [cited by applicant]
International Search Report and Written Opinion dated Oct. 25, 2019, issued in PCTUS2019045129. [cited by applicant]
International Search Report and Written Opinion mailed Feb. 3, 2021 issued in PCTUS2020059988. [cited by applicant]
Lassen, Bianca, et al. “Automatic Segmentation of Lung Lobes From Chest CT Images Based on Fissures, Vessels, and Bronchi,” IEEE, Institute for Medical Image Computing, 32.2 (2012), pp. 560-563. [cited by applicant]
Law, Max WK, et al., “Three Dimensional Curvilinear Structure Detection Using Optimally Oriented Flux,” Department of Computer Science and Engineering, LNCS 5305, 2008, pp. 368-382. [cited by applicant]
Mansoor, Awais, et al., “Segmentation and Image Analysis of Abnormal Lungs at CT: Current Approaches, Challenges, and Future Trends,” RadioGraphics, vol. 35, No. 4, Jul.-Aug. 2015, pp. 1056-1076. [cited by applicant]
Meng, Qier, et al. “Airway Segmentation From 3D Chest CT Volumes Based on Volume of Interest Using Gradient Vector Flow,” Medical Imaging Technology, 36.3 (2018), 4 pages. [cited by applicant]
OBGYN Key, “Augmented Reality in Minimally Invasive Digestive Surgery,” https://obgynkey.com/augmented-reality-in-minimally-invasive-digestive-surgery/, Nov. 10, 2020, pp. 1-6. [cited by applicant]
Ochs, Robert A., et al. “Automated classification of lung bronchovascular anatomy in CT using AdaBoost,” Medical Image Analysis, 11.3, Jun. 2007, pp. 315-324. [cited by applicant]
Payer, Christian, et al. “Automated Integer Programming Based Separation of Arteries and Veins From Thoracic CT Images,” Medical Image Analysis, May 4, 2016, pp. 1-19. [cited by applicant]
Payer, Christian, et al. “Automatic Artery-Vein Separation From Thoracic CT Images Using Integer Programming,” Conference Paper, DOI: 10.1007/978-3-319-24571-3_5, Oct. 2015, 9 pages. [cited by applicant]
Pu, Jiantao, et al., “A Differential Geometric Approach to Automated Segmentation of Human Airway Tree,” IEEE Transactions on Medical Imaging, 30(2), Feb. 2011, pp. 1-36. [cited by applicant]
Rohlfing, Torsten, et al. “Evaluation of Atlas Selection Strategies For Atlas-Based Image Segmentation With Application to Confocal Microscopy Images of Bee Brains,” Neuroimage, Nov. 7, 2003, 32 Pages. [cited by applicant]
Shekhovtsov, Alexander, et al., “Maximum Persistency Via Iterative Relaxed Inference With Graphical Models,” Proceedings of The IEEE, 2017, pp. 14. [cited by applicant]
Shirk, Joseph D., et al. “Effect of 3-Dimensional Virtual Reality Models For Surgical Planning of Robotic-Assisted Partial Nephrectomy on Surgical Outcomes: A Randomized Clinical Trial,” JAMA Network Open, Sep. 18, 2019… [cited by applicant]
Soler, Luc, et al., “Patient Specific Anatomy: The New Area of Anatomy Based on Computer Science Illustrated on Liver,” Journal of Visualized Surgery, J Vis Surg 2015;1:21, 12 Pages. [cited by applicant]
Türetken, Engin, et al. “Reconstructing Curvilinear Networks Using Path Classifiers and 10 Integer Programming,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, pp. 1-22. [cited by applicant]
Turetken, Engin, et al. “Reconstructing Loopy Curvilinear Structures Using Integer Programming,” Proceedings of The IEEE Conference, 2013, pp. 1822-1825. [cited by applicant]
US Notice of Allowance dated Feb. 10, 2022 issued in U.S. Appl. No. 16/949,685. [cited by applicant]
US Notice of Allowance dated May 4, 2022 issued in U.S. Appl. No. 16/949,685. [cited by applicant]
Van Dongen, Evelien, et al., “Automatic Segmentation of Pulmonary Vasculature in Thoracic CT Scans With Local Thresholding and Airway Wall Removal,” 2010 IEEE International Symposium on Biomedical Imaging: From Nano to … [cited by applicant]
Van Ginneken, Bram, et al., “Robust Segmentation and Anatomical Labeling of The Airway Tree From Thoracic CT Scans,” International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Berl… [cited by applicant]
EP Extended European Search report dated Mar. 19, 2024 in EP Application No. 20888543.4. [cited by applicant]
IN Office Action dated Dec. 18, 2024 in IN Application No. 202217028374. [cited by applicant]
U.S. Non-Final Office Action dated Jan. 29, 2024 in U.S. Appl. No. 17/250,572. [cited by applicant]
Van Rikxoort, E., et al., “Automatic Segmentation of Pulmonary Segments From Volumetric Chest CT Scans,” IEEE Transactions on Medical Imaging, 2009, vol. 28 (4), pp. 621-630. [cited by applicant]