IP Library › Granted Patent US 12,412,297
Granted Patent B2
US 12,412,297 · App. 17/887,649 · Granted Sep 9, 2025

System and method for identifying and classifying a feature in an image of a subject

Inventors: Patrick A. Helm (Canton, MA); Jeffrey H. Siewerdsen (Baltimore, MD); Ali Uneri (Columbia, MD); Craig K. Jones (Reisterstown, MD); Yixuan Huang (Baltimore, MD); Xiaoxuan Zhang (Baltimore, MD)
Assignees: Medtronic Navigation, Inc.; The Johns Hopkins University
G06T7/73A61B90/37G06T5/50G06T7/0012G06T7/11G06T7/337G06V10/16G06V10/22G06V10/25G06V10/40G06V10/44G06V10/764G06V10/82A61B2090/376G06T2207/10116G06T2207/20081G06T2207/20084G06T2207/20212G06T2207/20221G06V2201/03G06V2201/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,412,297
App. No.
17/887,649
Granted
Sep 9, 2025
Kind
B2
Abstract

A method and system is disclosed for analyzing image data of a subject. The image data can be collected with an imaging system in a selected manner and/or motion. The image data may include selected overlap and be acquired with an imaging system that generates a plurality of perspectives for more than one location. An automatic system and method may then define or identify various features and/or allow for registration for alternative image data.

Claims (64)

1. A method of detecting and classifying a feature in a plurality of individual image projections generated with an imaging system, the method comprising:

acquiring a first plurality of individual image projections as a plurality of input images of a first view of a subject;

acquiring a second plurality of individual image projections as a plurality of input images of a second view of the subject;

determining in at least a first sub-plurality of the acquired first plurality of individual image projections whether the feature is in each image projection of the first sub-plurality of image projections of the acquired plurality of individual image projections;

determining in at least a second sub-plurality of the acquired second plurality of individual image projections whether the feature is in each image projection of the second sub-plurality of image projections of the acquired plurality of individual image projections;

concatenating a feature map of the first sub-plurality of the acquired first plurality of individual image projections;

concatenating a feature map of the second sub-plurality of the acquired second plurality of individual image projections;

pairing region proposals in the first sub-plurality of the acquired first plurality of individual image projections with the second sub-plurality of the acquired second plurality of individual image projections using regression and classification;

performing a convolutional neural-network classification of the paired region proposals;

outputting a determination of the position of the feature in each image projection; and

display on a display device the outputted position of the determined feature and the determined feature using the plurality of individual image projections of the subject;

wherein displaying the determined feature allow for confirmation, selection, identification, or planning of a procedure.

2. The method of claim 1 , wherein the first view is different than the second view.

3. The method of claim 2 , wherein the first view is a lateral view and the second view is an anterior-to-posterior view.

4. The method of claim 2 , wherein the determination process is a machine learning algorithm operable to recognize the feature in the input image.

5. The method of claim 2 , wherein acquiring each of the first and second plurality of individual image projections includes acquiring a set of individual slot image projections is acquired of a subject with an imaging head at a selected position relative to the subject;

wherein a filter including a plurality of slots causes the acquisition of the set of individual slot image projections simultaneously;

wherein each individual slot image projection of the set of individual slot image projections has a unique perspective of the subject relative to each other individual slot image projection.

6. The method of claim 5 , wherein the first plurality of individual image projections is acquired with the slot filter at an anterior-to-posterior perspective relative to the subject;

wherein the second plurality of individual image projections is acquired with the slot filter at lateral perspective relative to the subject.

7. The method of claim 6 , wherein acquiring each of the first and second plurality of individual image projections comprises:

providing instructions to the imaging system to move the imaging head to a plurality of positions relative to the subject; and

projecting x-rays through the subject to a detector to generate the plurality of the sets of individual slot image projections.

8. The method of claim 2 , further comprising:

combining the first plurality of individual image projections by stitching together selected individual slot image projections of the acquired plurality of the sets of individual slot image projections to generate a first combined image of the subject; and

combining the second plurality of individual image projections by stitching together selected individual slot image projections of the acquired plurality of the sets of individual slot image projections to generate a second combined image of the subject;

wherein outputting the determination of the position of the feature in each image projection includes labeling at least one of the first combined image or the second combined image.

9. The method of claim 8 , further comprising:

confirming an order of the classified regions in at least one of the first combined image or the second combined image.

10. The method of claim 2 , further comprising:

adjusting weights in layers of a convolutional machine learning system for the determination in at least the first sub-plurality of the acquired first plurality of individual image projections whether the feature is in each image projection of the first sub-plurality of image projections of the acquired plurality of individual image projections and the determination in at least the second sub-plurality of the acquired second plurality of individual image projections whether the feature is in each image projection of the second sub-plurality of image projections of the acquired plurality of individual image projections.

11. A system operable to detect and classify a feature in a plurality of individual image projections generated with an imaging system, comprising:

a processor module configured to execute instructions to:

acquire a first plurality of individual image projections as a plurality of input images of a first view of a subject;

acquire a second plurality of individual image projections as a plurality of input images of a second view of the subject;

determine in at least a first sub-plurality of the acquired first plurality of individual image projections whether the feature is in each image projection of the first sub-plurality of image projections of the acquired plurality of individual image projections;

determine in at least a second sub-plurality of the acquired second plurality of individual image projections whether the feature is in each image projection of the second sub-plurality of image projections of the acquired plurality of individual image projections;

concatenate a feature map of the first sub-plurality of the acquired first plurality of individual image projections;

concatenate a feature map of the second sub-plurality of the acquired second plurality of individual image projections;

pair region proposals in the first sub-plurality of the acquired first plurality of individual image projections with the second sub-plurality of the acquired second plurality of individual image projections using regression and classification;

perform a convolutional neural-network classification of the paired region proposals;

output a determination of the position of the feature in each image projection; and

display on a display device the outputted position of the determined feature and the determined feature using the plurality of individual image projections of the subject;

wherein displaying the determined feature allow for confirmation, selection, identification, or planning of a procedure.

12. The system of claim 11 , wherein the first view is acquired at a different than the second view relative to the subject.

13. The system of claim 12 , wherein the first view is a lateral view of the subject and the second view is an anterior-to-posterior view of the subject.

14. The system of claim 12 , wherein the processor module configured to execute instructions includes a machine learning algorithm operable to recognize the feature in the input image.

15. The system of claim 12 , wherein the processor module configured to execute instructions further includes instructions to:

acquire each of the first and second plurality of individual image projections includes acquiring a set of individual slot image projections is acquired of a subject with an imaging head at a selected position relative to the subject;

wherein a filter including a plurality of slots causes the acquisition of the set of individual slot image projections simultaneously;

wherein each individual slot image projection of the set of individual slot image projections has a unique perspective of the subject relative to each other individual slot image projection.

16. The system of claim 15 , wherein the first plurality of individual image projections is acquired with the slot filter at an anterior-to-posterior perspective relative to the subject;

wherein the second plurality of individual image projections is acquired with the slot filter at lateral perspective relative to the subject.

17. The system of claim 16 , wherein acquiring each of the first and second plurality of individual image projections comprises:

executing instructions to move an imaging head to a plurality of positions relative to the subject; and

project x-rays through the subject to a detector to generate the plurality of the sets of individual slot image projections.

18. The system of claim 12 , wherein the processor module is further operable to execute instructions to:

combine the first plurality of individual image projections by stitching together selected individual slot image projections of the acquired plurality of the sets of individual slot image projections to generate a first combined image of the subject; and

combine the second plurality of individual image projections by stitching together selected individual slot image projections of the acquired plurality of the sets of individual slot image projections to generate a second combined image of the subject;

wherein outputting the determination of the position of the feature in each image projection includes labeling at least one of the first combined image or the second combined image.

19. The system of claim 18 , wherein the processor module is further operable to execute instructions to:

confirm an order of the classified regions in at least one of the first combined image or the second combined image.

20. The system of claim 12 , wherein the processor module is further operable to execute instructions to:

adjust weights in layers of a convolutional machine learning system for the determination in at least the first sub-plurality of the acquired first plurality of individual image projections whether the feature is in each image projection of the first sub-plurality of image projections of the acquired plurality of individual image projections and the determination in at least the second sub-plurality of the acquired second plurality of individual image projections whether the feature is in each image projection of the second sub-plurality of image projections of the acquired plurality of individual image projections.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 17, 2022
From: SIEWERDSEN, JEFFREY H.; UNERI, ALI; JONES, CRAIG K.; HUANG, YIXUAN; ZHANG, XIAOXUAN
To: THE JOHNS HOPKINS UNIVERSITY
Reel/Frame 060836/0179 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 17, 2022
From: HELM, PATRICK A.
To: MEDTRONIC NAVIGATION, INC.
Reel/Frame 060836/0534 →
Continuity (2)
Provisional Application 63283762 · Nov 29, 2021
Related Publication 20230169646A1 · Jun 1, 2023
References Cited (176)
US 8238631B2 · Hartmann et al. · 2012 [cited by applicant]
US 10881371B2 · Helm et al. · 2021 [cited by applicant]
US 11071507B2 · Helm et al. · 2021 [cited by applicant]
US 11138768B2 · Helm et al. · 2021 [cited by applicant]
US 20100290690A1 · Hartmann et al. · 2010 [cited by applicant]
US 20120099772A1 · Helm et al. · 2012 [cited by applicant]
US 20120250822A1 · Helm et al. · 2012 [cited by applicant]
US 20170231713A1 · Siewerdsen et al. · 2017 [cited by applicant]
US 20190030370A1 · Hibbard · 2019 [cited by applicant]
US 20190311505A1 · Helm et al. · 2019 [cited by applicant]
US 20200085382A1 · Taerum · 2020 [cited by examiner]
US 20200205763A1 · Helm et al. · 2020 [cited by applicant]
US 20200205764A1 · Helm et al. · 2020 [cited by applicant]
US 20210077047A1 · Tolkowsky · 2021 [cited by applicant]
US 20210397254A1 · Seibel et al. · 2021 [cited by applicant]
US 20220207722A1 · Kim · 2022 [cited by examiner]
US 20220309675A1 · Conjeti et al. · 2022 [cited by applicant]
US 20240307122A1 · Tolkowsky et al. · 2024 [cited by applicant]
WO 2023096831A1 · 2023 [cited by applicant]
WO 2023096834A1 · 2023 [cited by applicant]
WO 2023096835A1 · 2023 [cited by applicant]
WO 2023096836A1 · 2023 [cited by applicant]
Schmidt, S., Kappes, J., Bergtholdt, M., Pekar, V., Dries, S., Bystrov, D. and Schnorr., C., “Spine Detection and Labeling Using a Parts-Based Graphical Model,” Bienn. Int. Conf. Inf. Process. Med. Imaging, 122-133 (200… [cited by applicant]
Uneri, A., De Silva, T., Stayman, J. W., Kleinszig, G., Vogt, S., Khanna, A. J., Gokaslan, Z. L., Wolinsky, J. P. and Siewerdsen, J. H., “Known-component 3D-2D registration for quality assurance of spine surgery pedicle… [cited by applicant]
Weinstein, S. L., Dolan, L. A., Cheng, J. C. Y., Danielsson, A. and Morcuende, J. A., “Adolescent idiopathic scoliosis Stuart,” Lancet 371, 1527-1537 (2008). [cited by applicant]
Murphy, R. F. and Mooney, J. F., “Complications following spine fusion for adolescent idiopathic scoliosis,” Curr. Rev. Musculoskelet. Med. 9(4), 462-469 (2016). [cited by applicant]
Tonetti, J., Boudissa, M., Kerschbaumer, G. and Seurat, O., “Role of 3D intraoperative imaging in orthopedic and trauma surgery,” Orthop. Traumatol. Surg. Res. 106(1), S19-S25 (2020). [cited by applicant]
Penney, G. P., Little, J. A., Weese, J., Hill, D. L. G. and Hawkes, D. J., “Deforming a Preoperative Volume to Represent the Intraoperative Scene,” Comput. Aided Surg. 7(2), 63-73 (2002). [cited by applicant]
Park, S., Plishker, W., Quon, H., Wong, J., Shekhar, R. and Lee, J., “Deformable registration of CT and cone-beam CT with local intensity matching,” Phys. Med. Biol. 62(3), 927-947 (2017). [cited by applicant]
Sharp, G. C., Kandasamy, N., Singh, H. and Folkert, M., “GPU-based streaming architectures for fast cone-beam CT image reconstruction and demons deformable registration,” Phys. Med. Biol. 52(19), 5771-5783 (2007). [cited by applicant]
Nithiananthan, S., Schafer, S., Uneri, A., Mirota, D. J., Stayman, J. W., Zbijewski, W., Brock, K. K., Daly, M. J., Chan, H., Irish, J. C. and Siewerdsen, J. H., “Demons deformable registration of CT and cone-beam CT us… [cited by applicant]
Pei, Y., Zhang, Y., Qin, H., Ma, G., Guo, Y., Xu, T. and Zha, H., “Non-rigid Craniofacial 2D-3D Registration Using CNN-Based Regression,” Int. Work. Deep Learn. Med. Image Anal., 117-125 (2017). [cited by applicant]
Miao, S., Wang, Z. J. and Liao, R., “A CNN Regression Approach for Real-Time 2D/3D Registration,” IEEE Trans. Med. Imaging 35(5), 1352-1363 (2016). [cited by applicant]
Luckner, C., Herbst, M., Weber, T., Beister, M., Ritschl, L., Kappler, S. and Maier, A., “High-speed slot-scanning radiography using small-angle tomosynthesis: Investigation of spatial resolution,” Med. Phys. 46(12), 54… [cited by applicant]
Uneri, A., Zhang, X., Yi, T., Stayman, J. W., Helm, P. A., Theodore, N. and Siewerdsen, J. H., “Image quality and dose characteristics for an O-arm intraoperative imagingsystem with model-based image reconstruction,” Me… [cited by applicant]
Otake, Y., Wang, A. S., Webster Stayman, J., Uneri, A., Kleinszig, G., Vogt, S., Khanna, A. J., Gokaslan, Z. L. and Siewerdsen, J. H., “Robust 3D-2D image registration: application to spine interventions and vertebral l… [cited by applicant]
Uneri, A., Zhang, X., Siewerdsen, J. H., Theodore, N., Helm, P. A., Stayman, J. W. and Osgood, G. M., “3D-2D image registration in virtual long-film imaging: application to spinal deformity correction,” 52 (2019). [cited by applicant]
Penney, G. P., Weese, J., Little, J. a, Desmedt, P., Hill, D. L. and Hawkes, D. J., “A comparison of similarity measures for use in 2-D-3-D medical image registration.,” IEEE Trans. Med. Imaging 17(4), 586-595 (1998). [cited by applicant]
Uneri, A., Goerres, J., Silva, T. De, Jacobson, M. W., Ketcha, M. D., Reaungamorrat, S., Kleinszig, G., Vogt, S., Khanna, A. J., Wolinsky, J. P. and Siewerdsen, J. H., “Deformable 3D-2D registration of known components … [cited by applicant]
Uneri, A., Wu, P., Jones, C. K., Ketcha, M. D., Vagdargi, P., Han, R., Helm, P. A., Luciano, M., Anderson, W. S. and Siewerdsen, J. H., “Data-driven deformable 3D-2D registration for guiding neuroelectrode placement in … [cited by applicant]
Otake, Y., Wang, A. S., Uneri, A., Kleinszig, G., Vogt, S., Aygun, N., Sheng-fu, L. L., Wolinsky, J.-P., Gokaslan, Z. L. and Siewerdsen, J. H., “3D-2D registration in mobile radiographs: algorithm development and prelim… [cited by applicant]
Rampersaud, Y. R., Simon, D. A. and Foley, K. T., “Accuracy requirements for image-guided spinal pedicle screw placement,” Spine (Phila. Pa. 1976). 26(4), 352-359 (2001). [cited by applicant]
Ren, S., He, K., Girshick, R. and Sun, J., “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137-1149 (2017). [cited by applicant]
Nguyen, N. T., Tran, D. Q., Nguyen, N. T., & Nguyen, H. Q. (2020). A CNN-LSTM Architecture for Detection of Intracranial Hemorrhage on CT scans. arXiv preprint arXiv:2005.10992. [cited by applicant]
Pauly, O., Glocker, B., Criminisi, A., Mateus, D., Möller, A. M., Nekolla, S., & Navab, N. (Sep. 2011). Fast multiple organ detection and localization in whole-body MR Dixon sequences. In International Conference on Med… [cited by applicant]
Sa, R., Owens, W., Wiegand, R., Studin, M., Capoferri, D., Barooha, K., . . . & Chaudhary, V. (Jul. 2017). Intervertebral disc detection in X-ray images using faster R-CNN. In 2017 39th Annual International Conference o… [cited by applicant]
Sekuboyina, Anjany, et al. “Labeling Vertebrae with Two-dimensional Reformations of Multidetector CT Images: An Adversarial Approach for Incorporating Prior Knowledge of Spine Anatomy.” Radiology: Artificial Intelligenc… [cited by applicant]
Thian, Y. L., Li, Y., Jagmohan, P., Sia, D., Chan, V. E. Y., & Tan, R. T. (2019). Convolutional neural networks for automated fracture detection and localization on wrist radiographs. Radiology: Artificial Intelligence,… [cited by applicant]
Uhercík, Marián, et al. “Model fitting using RANSAC for surgical tool localization in 3-D ultrasound images.” IEEE Transactions on Biomedical Engineering 57.8 (2010): 1907-1916. [cited by applicant]
Uneri A, De Silva T, Goerres J, Jacobson MW, Ketcha MD, Reaungamorrat S, Kleinszig G, Vogt S, Khanna AJ, Osgood GM, Wolinsky JP, Siewerdsen JH. Intraoperative evaluation of device placement in spine surgery using known-… [cited by applicant]
Luo, X., Pu, Z., Xu, Y., Wong, W. Y., Su, J., Dou, X., Ye, B., Hu, J., & Mou, L. (2021). MVDRNet: Multi-view diabetic retinopathy detection by combining DCNNs and attention mechanisms. Pattern Recognition, 120, 108104. … [cited by applicant]
Ma, J., Li, X., Li, H., Wang, R., Menze, B. H., & Zheng, W. (2021). Cross-View Relation Networks for Mammogram Mass Detection. International Conference on Pattern Recognition. https://doi.org/10.1109/icpr48806.2021.9413… [cited by applicant]
Manbachi, A., De Silva, T., Uneri, A., Jacobson, M. P., Goerres, J., Ketcha, M. D., Han, R., Aygun, N., Thompson, D. R., Ye, X., Vogt, S., Kleinszig, G., Molina, C. A., Iyer, R. R., Garzon-Muvdi, T., Raber, M., Groves, … [cited by applicant]
Mody, M. G., Nourbakhsh, A., Stahl, D. L., Gibbs, M., Alfawareh, M., & Garges, K. J. (2008). The Prevalence of Wrong Level Surgery Among Spine Surgeons. Spine, 33(2), 194-198. https://doi.org/10.1097/brs.0b013e31816043d… [cited by applicant]
Nakai, S., Yoshizawa, H., & Kobayashi, S. (1999). Long-Term Follow-Up Study of Posterior Lumbar Interbody Fusion. Journal of Spinal Disorders, 12(4), 293???299. https://doi.org/10.1097/00002517-199908000-00004. [cited by applicant]
Panjabi, M. M., Duranceau, J., Goel, V. K., Oxland, T. R., & Takata, K. (1991). Cervical Human Vertebrae Quantitative Three-Dimensional Anatomy of the Middle and Lower Regions. Spine, 16(8), 861-869. https://doi.org/10.… [cited by applicant]
Panjabi, M. M., Takata, K., Goel, V. K., Federico, D. J., Oxland, T. R., Duranceau, J., & Krag, M. H. (1991). Thoracic Human Vertebrae Quantitative Three-Dimensional Anatomy. Spine, 16(8), 888-901. https://doi.org/10.10… [cited by applicant]
Panjabi, M. M., Goel, V. K., Oxland, T. R., Takata, K., Duranceau, J., Krag, M. H., & Price, M. (1992). Human Lumbar Vertebrae. Spine, 17(3), 299-306. https://doi.org/10.1097/00007632-199203000-00010. [cited by applicant]
Qin, C., Yao, D., Zhuang, H. R., Wang, H., Shi, Y., & Song, Z. (2019). Residual Block-based Multi-Label Classification and Localization Network with Integral Regression for Vertebrae Labeling. ArXiv (Cornell University)… [cited by applicant]
Rasoulian, A., Rohling, R., & Abolmaesumi, P. (2013). Lumbar Spine Segmentation Using a Statistical Multi-Vertebrae Anatomical Shape+Pose Model. IEEE Transactions on Medical Imaging, 32(10), 1890-1900. https://doi.org/1… [cited by applicant]
Rubin, J. M., Sanghavi, D., Claire, Z., Lee, K. L., Qadir, A., & Xu-Wilson, M. (2018). Large Scale Automated Reading of Frontal and Lateral Chest X-Rays using Dual Convolutional Neural Networks. ArXiv: Computer Vision a… [cited by applicant]
Scholtz, J., Wichmann, J. L., Kaup, M., Fischer, S., Kerl, J. M., Lehnert, T., Vogl, T. J., & Bauer, R. W. (2015). First performance evaluation of software for automatic segmentation, labeling and reformation of anatomi… [cited by applicant]
Takami, M., Taiji, R., Tsutsui, S., Iwasaki, H., Okada, M., Minamide, A., Yukawa, Y., Hashizume, H., & Yamada, H. (2021). Impact of an intraoperative coronal spinal alignment measurement technique using a navigational t… [cited by applicant]
Tran, V. H., Lin, H., & Liu, H. (2020). MBNet: A Multi-task Deep Neural Network for Semantic Segmentation and Lumbar Vertebra Inspection on X-Ray Images. Lecture Notes in Computer Science, 635-651. https://doi.org/10.10… [cited by applicant]
Van Tulder, G., Tong, Y., & Marchiori, E. (2021). Multi-view Analysis of Unregistered Medical Images Using Cross-View Transformers. Lecture Notes in Computer Science, 104-113. https://doi.org/10.1007/978-3-030-87199-4_1… [cited by applicant]
Uneri, A., Wang, S. A., Otake, Y., Kleinszig, G., Teske, J., Khanna, A. J., Gallia, G. L., Gokaslan, Z. L., & Siewerdsen, J. H. (2014). Evaluation of low-dose limits in 3D-2D rigid registration for surgical guidance. Ph… [cited by applicant]
Vaynrub, M., Hirsch, B. E., Tishelman, J. C., Vasquez-Montes, D., Buckland, A. J., Errico, T. J., & Protopsaltis, T. S. (2018). Validation of prone intraoperative measurements of global spinal alignment. Journal of Neur… [cited by applicant]
Wang, F., Zheng, K., Lu, L., Xiao, J., Wu, M., & Miao, S. (2021). Automatic Vertebra Localization and Identification in CT by Spine Rectification and Anatomically-constrained Optimization. Computer Vision and Pattern Re… [cited by applicant]
Wang, L., Xu, Q., Leung, S., Chung, J. H., Chen, B., & Li, S. (2019). Accurate automated Cobb angles estimation using multi-view extrapolation net. Medical Image Analysis, 58, 101542. https://doi.org/10.1016/j.media.201… [cited by applicant]
Wang, L., Traub, J., Heining, S. M., Benhimane, S., Euler, E., Graumann, R., & Navab, N. (2008). Long Bone X-Ray Image Stitching Using Camera Augmented Mobile C-Arm. Lecture Notes in Computer Science, 578-586. https://d… [cited by applicant]
Watanabe, K., Aoki, Y., & Matsumoto, M. (2019). An Application of Artificial Intelligence to Diagnostic Imaging of Spine Disease: Estimating Spinal Alignment From Moire Images. Neurospine, 16(4), 697-702. https://doi.or… [cited by applicant]
Weng, C., Wang, T., Huang, Y., Yeh, Y., Fu, C., Yeh, C., & Tsai, T. (2019). Artificial Intelligence for Automatic Measurement of Sagittal Vertical Axis Using ResUNet Framework. Journal of Clinical Medicine, 8(11), 1826.… [cited by applicant]
International Preliminary Report on Patentability related to International Application No. PCT/US2022/050406 dated May 2, 2024, 13 pages. [cited by applicant]
International Preliminary Report on Patentability related to International Application No. PCT/US2022/050422 dated May 2, 2024, 12 pages. [cited by applicant]
International Preliminary Report on Patentability related to International Application No. PCT/US2022/050414 dated May 2, 2024, 14 pages. [cited by applicant]
International Preliminary Report on Patentability related to International Application No. PCT/US2022/050400 dated May 2, 2024, 15 pages. [cited by applicant]
Huang, Y., Uneri, A., Jones, C. K., Zhang, X., Ketcha, M. D., Aygun, N., Helm, P. A. and Siewerdsen, J. H., “3D vertebrae labeling in spine CT: An accurate, memory-efficient (Ortho2D) framework,” Phys. Med. Biol. 66(12)… [cited by applicant]
Klinder, T., Ostermann, J., Ehm, M., Franz, A., Kneser, R. and Lorenz, C., “Automated model-based vertebra detection, identification, and segmentation in CT images,” Med. Image Anal. 13(3), 471-482 (2009). [cited by applicant]
Chen, Y., Gao, Y., Li, K., Zhao, L. and Zhao, J., “Vertebrae Identification and Localization Utilizing Fully Convolutional Networks and a Hidden Markov Model,” IEEE Trans. Med. Imaging 39(2), 387-399 (2020). [cited by applicant]
Ailon, T., Smith, J. S., Shaffrey, C. I., Lenke, L. G., Brodke, D., Harrop, J. S., Fehlings, M. and Ames, C. P., “Degenerative spinal deformity,” Neurosurgery 77(4), S75-S91 (2015). [cited by applicant]
Betz, R. R., Kim, J., D'Andrea, L. P., Mulcahey, M. J., Balsara, R. K. and Clements, D. H., “An innovative technique of vertebral body stapling for the treatment of patients with adolescent idiopathic scoliosis: A feasi… [cited by applicant]
Diebo, B. G., Shah, N. V., Boachie-Adjei, O., Zhu, F., Rothenfluh, D. A., Paulino, C. B., Schwab, F. J. and Lafage, V., “Adult spinal deformity,” Lancet 394(10193), 160-172 (2019). [cited by applicant]
Jain, A., Hassanzadeh, H., Puvanesarajah, V., Klineberg, E. O., Sciubba, D. M., Kelly, M. P., Hamilton, D. K., Lafage, V., Buckland, A. J., Passias, P. G., Protopsaltis, T. S., Lafage, R., Smith, J. S., Shaffrey, C. I. … [cited by applicant]
Hu, X., Ohnmeiss, D. D. and Lieberman, I. H., “Robotic-assisted pedicle screw placement: Lessons learned from the first 102 patients,” Eur. Spine J. 22(3), 661-666 (2013). [cited by applicant]
Little, J. A., Hill, D. L. G. and Hawkes, D. J., “Deformations Incorporating Rigid Structures,” Comput. Vis. Image Underst. 66(2), 223-232 (1997). [cited by applicant]
Kohlrausch, J., Rohr, K. and Stiehl, H. S., “A new class of elastic body splines for nonrigid registration of medical images,” J. Math. Imaging Vis. 23(3), 253-280 (2005). [cited by applicant]
Brock, K. K., Sharpe, M. B., Dawson, L. A., Kim, S. M. and Jaffray, D. A., “Accuracy of finite element model-based multi-organ deformable image registration,” Med. Phys. 32(6), 1647-1659 (2005). [cited by applicant]
Gill, S., Abolmaesumi, P., Fichtinger, G., Boisvert, J., Pichora, D., Borshneck, D. and Mousavi, P., “Biomechanically constrained groupwise ultrasound to CT registration of the lumbar spine,” Med. Image Anal. 16(3), 662… [cited by applicant]
Lawson, J. D., Schreibmann, E., Jani, A. B. and Fox, T., “Quantitative evaluation of a cone-beam computed tomography-planning computed tomography deformable image registration method for adaptive radiation therapy,” J. … [cited by applicant]
Kearney, V., Haaf, S., Sudhyadhom, A., Valdes, G. and Solberg, T. D., “An unsupervised convolutional neural network-based algorithm for deformable image registration,” Phys. Med. Biol. 63(18) (2018). [cited by applicant]
Eppenhof, K. A. J. and Pluim, J. P. W., “Pulmonary CT Registration Through Supervised Learning With Convolutional Neural Networks,” IEEE Trans. Med. Imaging 38(5), 1097-1105 (2019). [cited by applicant]
Esfandiari, H., Newell, R., Anglin, C., Street, J. and Hodgson, A. J., “A deep learning framework for segmentation and pose estimation of pedicle screw implants based on C-arm fluoroscopy,” Int. J. Comput. Assist. Radio… [cited by applicant]
Ladd, B., Jones, K. and Polly, D., “2-Dimensional Long Film O-Arm Imaging, an Alternative When Intraoperative Fluoroscopy Is Inadequate,” World Neurosurg. 150, 54-55 (2021). [cited by applicant]
Ketcha, M. D., De Silva, T., Uneri, A., Jacobson, M. W., Goerres, J., Kleinszig, G., Vogt, S., Wolinsky, J. P. and Siewerdsen, J. H., “Multi-stage 3D-2D registration for correction of anatomical deformation in image-gui… [cited by applicant]
Helm, P. A., Teichman, R., Hartmann, S. L. and Simon, D., “Spinal Navigation and Imaging: History, Trends, and Future,” IEEE Trans. Med. Imaging 34(8), 1738-1746 (2015). [cited by applicant]
De Silva, T., Uneri, A., Ketcha, M. D., Reaungamornrat, S., Kleinszig, G., Vogt, S., Aygun, N., Lo, S. F., Wolinsky, J. P. and Siewerdsen, J. H., “3D-2D image registration for target localization in spine surgery: inves… [cited by applicant]
Jansen, N. and Ostermeier, A., “Completely derandomized self-adaptation in evolution strategies.,” Evol. Comput. 9(2), 159-195 (2001). [cited by applicant]
Ketcha, M. D., De Silva, T., Uneri, A., Jacobson, M. W., Goerres, J., Kleinszig, G., Vogt, S., Wolinsky, J. P. and Siewerdsen, J. H., “Multi-stage 3D-2D registration for correction of anatomical deformation in image-gui… [cited by applicant]
Hartley, R., Trumpf, J., Dai, Y. and Li, H., “Rotation averaging,” Int. J. Comput. Vis. 103(3), 267-305 (2013). [cited by applicant]
Been, E. and Kalichman, L., “Lumbar lordosis,” Spine J. 14(1), 87-97 (2014). [cited by applicant]
Johnson, H., Harris, G. and Williams, K., “BRAINSFit: mutual information rigid registrations of whole-brain 3D images, using the insight toolkit,” Insight J 57(1) (2007). [cited by applicant]
Li, P., Chen, X. and Shen, S., “Stereo R-CNN based 3D Object Detection for Autonomous Driving,” Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 7644-7652 (2019). [cited by applicant]
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B. and Belongie, S., “Feature Pyramid Networks for Object Detection” (2016). [cited by applicant]
Glocker, B., Feulner, J., Criminisi, A., Haynor, D. R. and Konukoglu, E., “Automatic localization and identification of vertebrae in arbitrary field-of-view CT scans,” Lect. Notes Comput. Sci. (including Subser. Lect. N… [cited by applicant]
Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moore, S., Phillips, S., Maffitt, D., Pringle, M., Tarbox, L. and Prior, F., “The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public I… [cited by applicant]
Ketcha, M. D., De Silva, T., Han, R., Uneri, A., Vogt, S., Kleinszig, G. and Siewerdsen, J. H., “A Statistical Model for Rigid Image Registration Performance: The Influence of Soft-Tissue Deformation as a Confounding No… [cited by applicant]
“A Pedestrian Detection Model Based on Binocular Information Fusion https://ieeexplore.ieee.org/document/8770601”. [cited by applicant]
“Aerial Target Tracking Algorithm Based on Faster R-CNN Combined with Frame Differencing https://www.mdpi.com/2226-4310/4/2/32/pdf”. [cited by applicant]
Akselrod-Ballin, A., Karlinsky, L., Alpert, S., Hasoul, S., Ben-Ari, R., & Barkan, E. (2016). A region based convolutional network for tumor detection and classification in breast mammography. In Deep learning and data … [cited by applicant]
“Automatic Localization of the Left Ventricle from Cardiac Cine Magnetic Resonance Imaging: A New Spectrum-Based Computer-Aided Tool https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3982962/”. [cited by applicant]
Bouget, D., Allan, M., Stoyanov, D., & Jannin, P. (2017). Vision-based and marker-less surgical tool detection and tracking: a review of the literature. Medical image analysis, 35, 633-654. [cited by applicant]
Cao, Z., Duan, L., Yang, G., Yue, T., Chen, Q., Fu, H., & Xu, Y. (Sep. 2017). Breast tumor detection in ultrasound images using deep learning. In International Workshop on Patch-based Techniques in Medical Imaging (pp. … [cited by applicant]
“Convolutional Neural Networks: A Binocular Vision Perspective https://arxiv.org/pdf/1912.10201v1.pdf”. [cited by applicant]
Criminisi, A., Shotton, J., Robertson, D., & Konukoglu, E. (Sep. 2010). Regression forests for efficient anatomy detection and localization in CT studies. In International MICCAI Workshop on Medical Computer Vision (pp.… [cited by applicant]
“Deep Learning for Cardiac Image Segmentation: A Review https://www.frontiersin.org/articles/10.3389/fcvm.2020.00025/full”. [cited by applicant]
“Deep Object Co-Segmentation https://arxiv.org/abs/1804.06423”. [cited by applicant]
“Deep Sequential Segmentation of Organs in Volumetric Medical Scans https://arxiv.org/pdf/1807.02437.pdf”. [cited by applicant]
Doerr, S. A., et al. “Data-driven detection and registration of spine surgery instrumentation in intraoperative images.” Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling. vol. 11315. In… [cited by applicant]
Dong, Hao, et al. “Automatic brain tumor detection and segmentation using u-net based fully convolutional networks.” annual conference on medical image understanding and analysis. Springer, Cham, 2017. [cited by applicant]
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks https://ieeexplore.ieee.org/document/7485869. [cited by applicant]
Forsberg, Daniel, Erik Sjoblom, and Jeffrey L. Sunshine. “Detection and labeling of vertebrae in MR images using deep learning with clinical annotations as training data.” Journal of digital imaging 30.4 (2017): 406-412. [cited by applicant]
Gao, C., Unberath, M., Taylor, R., & Armand, M. (2019). Localizing dexterous surgical tools in X-ray for image-based navigation. arXiv preprint arXiv:1901.06672. [cited by applicant]
Ghesu, F. C., Georgescu, B., Zheng, Y., Grbic, S., Maier, A., Hornegger, J., & Comaniciu, D. (2017). Multi-scale deep reinforcement learning for real-time 3D-landmark detection in CT scans. IEEE transactions on pattern … [cited by applicant]
Gooßen, A., Deshpande, H., Harder, T., Schwab, E., Baltruschat, I., Mabotuwana, T., . . . & Saalbach, A. (2019). Deep learning for pneumothorax detection and localization in chest radiographs. arXiv preprint arXiv:1907.… [cited by applicant]
“Group-wise Deep Object Co-Segmentation with Co-Attention Recurrent Neural Network https://openaccess.thecvf.com/content_ICCV_2019/papers/Li_Group-Wise_Deep_Object_Co-Segmentation_With_Co-Attention_Recurrent_Neural_Netw… [cited by applicant]
Hasan, S. M., & Linte, C. A. (2019). U-NetPlus: a modified encoder-decoder U-Net architecture for semantic and instance segmentation of surgical instrument. arXiv preprint arXiv:1902.08994. [cited by applicant]
Li Peiliang et al: “ Stereo R-CNN Based 3D Object Detection for Autonomous Driving”, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition ( CVPR), IEEE, Jun. 15, 2019 ( Jun. 15, 2019), pp. 7636-7644. [cited by applicant]
Zhang Xiaoxuan et al: “Deformable 3D-2D 1-20 image registration and analysis of global spinal alignment in long-length intraoperative spine imaging”, Medical Physics., vol. 49, No. 9, Jul. 27, 2022 (Jul. 27, 2022), pp. … [cited by applicant]
Uneri Ali et al: “Multi-slot intraoperative imaging and 3D-2D registration for evaluation of long surgical constructs in spine surgery”, Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling… [cited by applicant]
Uneri A et al: “Intraoperative evaluation of device placement in spine surgery using known-component 30-20⋅image registration”, Physics in Medicine and Biology, Institute of Physics Publishing, Bristol GB, vol. 62,. No.… [cited by applicant]
Drobny David et al: “Towards Automated Spine Mobility Quantification: A Locally Rigid CT to X-ray Registration Framework”, Jun. 9, 2020 (Jun. 9. 2020), 16th European Conference—Computer Vision—ECCV 2020, Cornell Univers… [cited by applicant]
International Search Report and Written Opinion regarding International Patent Application No. PCT/US2022/050414, dated Mar. 9, 2023. [cited by applicant]
Zhang Xiaoxuan et al: “Long-length tomosynthesis and 3D-2D registration for intraoperative assessment of spine instrumentation”, Physics in Medicine and Biology, Institute of Physics Publishing, Bristol, GB, vol. 66,. N… [cited by applicant]
International Search Report and Written Opinion regarding International Patent Application No. PCT/US2022/050422, dated Mar. 10, 2023. [cited by applicant]
International Search Report and Written Opinion regarding International Patent Application No. PCT/US2022/050406, dated Mar. 9, 2023. [cited by applicant]
Xie, Y., Chao, M., Lee, P. and Xing, L., “Feature-based rectal contour propagation from planning CT to cone beam CT,” Med. Phys. 35(10), 4450-4459 (2008). [cited by applicant]
Zhen, X., Yan, H., Zhou, L., Jia, X. and Jiang, S. B., “Deformable image registration of CT and truncated cone-beam CT for adaptive radiation therapy,” Phys. Med. Biol. 58(22), 7979-7993 (2013). [cited by applicant]
Zhang, X., Uneri, A., Wu, P., Ketcha, M. D., Jones, C. K., Huang, Y., Lo, S. F. L., Helm, P. A. and Siewerdsen, J. H., “Long-length tomosynthesis and 3D-2D registration for intraoperative assessment of spine instrumenta… [cited by applicant]
Zhang, J., Weir, V., Fajardo, L., Lin, J., Hsiung, H. and Ritenour, E. R., “Dosimetric characterization of a cone-beam O-armTM imaging system,” J. Xray. Sci. Technol. 12(4), 305-317 (2009). [cited by applicant]
Wozniak, M., Sitka, J., & Wieczorek, M. (2021). Deep neural network correlation learning mechanism for CT brain tumor detection. Neural Computing and Applications, 1-16. [cited by applicant]
Yamazaki, Y., Kanaji, S., Matsuda, T., Oshikiri, T., Nakamura, T., Suzuki, S., . . . & Kakeji, Y. (2020). Automated surgical instrument detection from laparoscopic gastrectomy video images using an open source convoluti… [cited by applicant]
Zhang, R., Cheng, C., Zhao, X., & Li, X. (2019). Multiscale mask R-CNN-based lung tumor detection using PET Imaging. Molecular imaging, 18, 1536012119863531. [cited by applicant]
Zhang, X., Uneri, A., Wu, P., Ketcha, M. D., Doerr, S. A., Jones, C. K., Helm, P. A., & Siewerdsen, J. H. (2020). Multi-slot extended view imaging on the O-Arm: image quality and application to intraoperative assessment… [cited by applicant]
Wu, H., Bailey, C., Rasoulinejad, P., & Li, S. (2018). Automated comprehensive Adolescent Idiopathic Scoliosis assessment using MVC-Net. Medical Image Analysis, 48, 1-11. https://doi.org/10.1016/j.media.2018.05.005. [cited by applicant]
Yang, D. H., Xiong, T., & Xu, D. (2018). Automatic Vertebra Labeling in Large-Scale Medical Images Using Deep Image-to-Image Network with Message Passing and Sparsity Regularization. Advances in Computer Vision and Patt… [cited by applicant]
Yang, Z., Cao, Z., Zhang, Y., Tang, Y., Lin, X., Ouyang, R., Wu, M., Han, M. K., Xiao, J., Huang, L., Wu, S., Chang, P. Y., & Ma, J. (2021). MommiNet-v2: Mammographic multi-view mass identification networks. Medical Ima… [cited by applicant]
Yeh, Y., Weng, C., Huang, Y., Fu, C., Tsai, T., & Yeh, C. (2021). Deep learning approach for automatic landmark detection and alignment analysis in whole-spine lateral radiographs. Scientific Reports, 11(1). https://doi… [cited by applicant]
Huang Y. et al: “Multi-perspective region-based CNNs for vertebrae labeling in intraoperative long-length images”, Computer Methods and Programs in Biomedicine., vol. 227, Nov. 3, 2022 (Nov. 3, 2022), p. 107222. [cited by applicant]
Haofu Liao et al: “Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information”, arxiv.org, Cornell University Library, 201 Olin Library Cornell Universi… [cited by applicant]
Zhang Kailai et al: “MPF-net: An effective framework for automated cobb angle estimation”, Medical Image Analysis, Oxford University Press, Oxofrd, GB, vol. 75, Oct. 16, 2021 (Oct. 16, 2021). [cited by applicant]
International Search Report and Written Opinion regarding International Application No. PCT/US2022/050400, dated Mar. 2, 2023. [cited by applicant]
“Image Co-Segmentation Using Graph Convolution Neural Network https://www.ee.iitb.ac.in/course/˜avik/ICVGIP18_arXiv.pdf”. [cited by applicant]
Jin, Y., Dou, Q., Chen, H., Yu, L., Qin, J., Fu, C. W., & Heng, P. A. (2017). SV-RCNet: workflow recognition from surgical videos using recurrent convolutional network. IEEE transactions on medical imaging, 37(5), 1114-… [cited by applicant]
Kim, K. C., Cho, H. C., Jang, T. J., Choi, J. M., & Seo, J. K. (2020). Automatic detection and segmentation of lumbar vertebrae from X-ray images for compression fracture evaluation. Computer Methods and Programs in Bio… [cited by applicant]
“Object Detection Based on Binocular Vision with Convolutional Neural Network https://doi.org/10.1145/3297067.3297081”. [cited by applicant]
“Object Detection in Multi-view X-Ray Images https://link.springer.com/chapter/10.1007/978-3-642-32717-9_15”. [cited by applicant]
“Semantic Segmentation of Video Sequences with Convolutional LSTMs https://arxiv.org/pdf/1905.01058.pdf”. [cited by applicant]
“Spatiotemporal CNN for Video Object Segmentation https://openaccess.thecvf.com/content_CVPR_2019/papers/Xu_Spatiotemporal_CNN_for_Video_Object_Segmentation_CVPR_2019_paper.pdf”. [cited by applicant]
“X-Net: A Binocular Summation Network for Foreground Segmentation https://doi.org/10.1109/ACCESS.2019.2919802”. [cited by applicant]
“You Only Look Once https://pjreddie.com/darknet/yolo/”. [cited by applicant]
Bayat, A., Sekuboyina, A., Hofmann, F., Husseini, M. E., Kirschke, J. S., & Menze, B. H. (2019). Vertebral Labelling in Radiographs: Learning a Coordinate Corrector to Enforce Spinal Shape. Springer International Publis… [cited by applicant]
Arif, S. M. M. R. A., Knapp, K. M., & Slabaugh, G. (2018). Fully automatic cervical vertebrae segmentation framework for X-ray images. Computer Methods and Programs in Biomedicine, 157, 95-111. https://doi.org/10.1016/j… [cited by applicant]
De Silva, T., Vedula, S. S., Perdomo-Pantoja, A., Vijayan, R. C., Doerr, S. A., Uneri, A., Han, R., Ketcha, M. D., Skolasky, R. L., Witham, T. F., Theodore, N., & Siewerdsen, J. H. (2020). SpineCloud: image analytics fo… [cited by applicant]
DeVine, J. G., Chutkan, N. B., Norvell, D. C., & Dettori, J. R. (2010). Avoiding Wrong Site Surgery. Spine, 35 (Supplement), S28-S36. https://doi.org/10.1097/brs.0b013e3181d833ac. [cited by applicant]
Diebo, B. G., Oren, J. H., Challier, V., Lafage, R., Ferrero, E., Liu, S., Vira, S., Spiegel, M., Harris, B. P., Liabaud, B., Henry, J. K., Errico, T. J., Schwab, F. J., & Lafage, V. (2016). Global sagittal axis: a step… [cited by applicant]
Doerr, S. A., De Silva, T., Vijayan, R. C., Han, R., Uneri, A., Ketcha, M. D., Zhang, X., Khanna, N., Westbroek, E. M., Jiang, B., Zygourakis, C. C., Aygun, N., Theodore, N., & Siewerdsen, J. H. (2020). Automatic analys… [cited by applicant]
Gilad, I., & Nissan, M. (1985). Sagittal radiographic measurements of the cervical and lumbar vertebrae in normal adults. British Journal of Radiology, 58(695), 1031-1034. https://doi.org/10.1259/0007-1285-58-695-1031. [cited by applicant]
Huang, Y., Uneri, A., Jones, C. K., Zhang, X., Ketcha, M. D., Aygun, N., Helm, P. A., & Siewerdsen, J. H. (2021). 3D vertebrae labeling in spine CT: an accurate, memory-efficient (Ortho2D) framework. Physics in Medicine… [cited by applicant]
Huang, Y., Jones, C. K., Zhang, X., Johnston, A., Aygun, N., Witham, T. F., Helm, P. A., Siewerdsen, J., & Uneri, A. (2022). Automatic labeling of vertebrae in long-length intraoperative imaging with a multi-view, regio… [cited by applicant]
Kurochka, K. S., & Panarin, K. A. (2021). An algorithm of segmentation of a human spine X-ray image with the help of Mask R-CNN neural network for the purpose of vertebrae localization. 2021 56th International Scientifi… [cited by applicant]
Lehman, R. A., Lenke, L. G., Helgeson, M. D., Eckel, T. T., & Keeler, K. A. (2010). Do Intraoperative Radiographs in Scoliosis Surgery Reflect Radiographic Result? Clinical Orthopaedics and Related Research, 468(3), 679… [cited by applicant]
Li, Y., Liang, W., Zhang, Y., An, H., & Tan, J. (2016). Automatic Lumbar Vertebrae Detection Based on Feature Fusion Deep Learning for Partial Occluded C-arm X-ray Images. International Conference of the IEEE Engineerin… [cited by applicant]
Liao, H., Mesfin, A., & Luo, J. (2018). Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information. IEEE Transactions on Medical Imaging, 37(5), 1266-12… [cited by applicant]
Illés, T., & Somoskeöy, S. (2012). The EOSTM imaging system and its uses in daily orthopaedic practice. International Orthopaedics, 36(7), 1325-1331. https://doi.org/10.1007/s00264-012-1512-y. [cited by applicant]
Cai, Yunliang, et al. “Multi-modality vertebra recognition in arbitrary views using 3D deformable hierarchical model.” IEEE transactions on medical imaging 34.8 (2015): 1676-1693. (Year: 2015). [cited by applicant]
Qin, Chunli, et al. “Vertebrae labeling via end-to-end integral regression localization and multi-label classification network.” IEEE Transactions on Neural Networks and Learning Systems 33.6 (2021): 2726-2736. (Year: 2… [cited by applicant]