IP Library › Granted Patent US 12,599,457
Granted Patent B1
US 12,599,457 · App. 18/773,817 · Granted Apr 14, 2026

Mixed reality endoscopic retrograde cholangiopancreatopgraphy (ERCP) procedure

Inventor: Navakanth Gorrepati (Southlake, TX)
A61B90/37A61B1/273A61B2090/365
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,599,457
App. No.
18/773,817
Granted
Apr 14, 2026
Kind
B1
Abstract

A mixed reality endoscopic surgical procedure method, including the steps of selecting a three-dimensional holographic first image of a body part to be surgically operated on and displaying the three-dimensional holographic first image of the body part. The method can further include positioning an endoscope relative to the body part, wherein the endoscope further includes a guidewire, displaying a three-dimensional second holographic image or video of the endoscope or guidewire relative to the body part, and using the three-dimensional first holographic first image as a visual guide for engaging the body part via the guidewire.

Claims (35)

1 . A mixed reality endoscopic surgical procedure method, comprising:

selecting a three-dimensional holographic first image or video representative of a body part to be surgically operated on;

displaying the three-dimensional holographic first image or video of the body part on a display;

positioning an endoscope comprising a camera relative to the body part, wherein the endoscope further comprises a guidewire;

displaying, from the camera of the endoscope, a second video of the guidewire relative to the body part;

overlaying the second video and the three-dimensional first holographic image or video over each other;

displaying one or more graphical markers on or adjacent to the overlaid second video and three-dimensional holographic first image or video, wherein the one or more graphical markers are adapted to visually assist a user to guide the guidewire; and

penetrating the body part via the guidewire.

2 . The mixed reality endoscopic surgical procedure method of claim 1 , wherein the first holographic first image is comprised of a duodenum illustrating a major papilla orifice or ampulla.

3 . The mixed reality endoscopic surgical procedure method of claim 1 , wherein the body part is comprised of the major papilla orifice or ampulla.

4 . The mixed reality endoscopic surgical procedure method of claim 1 , wherein the endoscope is further comprised of a sphincterotome.

5 . The mixed reality endoscopic surgical procedure method of claim 4 , further comprising:

orientating the sphincterotome to a desired position relative to the position of the first holographic image of the body part.

6 . The mixed reality endoscopic surgical procedure method of claim 4 , further comprising:

orientating the sphincterotome within a range of about 10-degrees to about 80-degrees relative to a horizontal plane and further with respect to the position of the first holographic image of the body part.

7 . The mixed reality endoscopic surgical procedure method of claim 1 , further comprising displaying a three-dimensional third holographic image or video relative to the three-dimensional first holographic image and the video.

8 . The mixed reality endoscopic surgical procedure method of claim 1 , further comprising displaying one or more holographic information of patient vitals.

9 . The mixed reality endoscopic surgical procedure method of claim 1 , wherein the video is further comprised of streaming one or more video or image data from a first source.

10 . The mixed reality endoscopic surgical procedure method of claim 1 , further comprising displaying one or more holographic images of one or more users and further communicating with the one or more users via audio or video.

11 . A mixed reality endoscopic surgical procedure method, comprising:

selecting a three-dimensional holographic first image representative of a body part to be surgically operated on;

displaying the three-dimensional holographic first image of the body part via an augmented reality or virtual reality headset;

positioning an endoscope comprising a camera relative to the body part, wherein the endoscope further comprises a guidewire;

displaying a second video, via a feed from the camera of the endoscope, the body part relative to the guidewire;

overlaying the three-dimensional first holographic image or video and the second video over each other;

displaying one or more visual representations on or adjacent to the overlaid second video and three-dimensional holographic first image or video; and

positioning the guidewire using the one or more visual representations and the overlaid second video and three-dimensional holographic first image or video.

12 . A mixed reality endoscopic surgical procedure method, comprising:

selecting a first image representative of a body part to be surgically operated on;

displaying the first image of the body part via an augmented reality or virtual reality headset;

positioning an endoscope comprising a camera relative to the body part, wherein the endoscope further comprises a catheter;

displaying a second image or video, retrieved via the camera of the endoscope, the body part relative to the catheter;

overlaying the first image and the second image or video over each other;

displaying one or more visual representations on or adjacent to the overlaid first image and second image or video; and

orienting the catheter using the one or more visual representations and the overlaid first image and second image or video.

Continuity (1)
Continuation 17399586 · Aug 11, 2021
References Cited (48)
US 5810807A · Ganz · 1998 [cited by examiner]
US 6346940B1 · Fukunaga · 2002 [cited by examiner]
US 10220181B2 · Giap et al. · 2019 [cited by applicant]
US 11094223B2 · Lampotang · 2021 [cited by examiner]
US 11523874B2 · Popovic · 2022 [cited by examiner]
US 12127890B1 · Gorrepati · 2024 [cited by examiner]
US 20010027272A1 · Saito · 2001 [cited by examiner]
US 20050033117A1 · Ozaki · 2005 [cited by examiner]
US 20080243142A1 · Gildenberg · 2008 [cited by examiner]
US 20090012390A1 · Pescatore · 2009 [cited by examiner]
US 20100057077A1 · Ducharme · 2010 [cited by examiner]
US 20130211246A1 · Parasher · 2013 [cited by applicant]
US 20140303491A1 · Shekhar · 2014 [cited by examiner]
US 20150126859A1 · Popovic · 2015 [cited by examiner]
US 20150141808A1 · Elhawary · 2015 [cited by examiner]
US 20150302634A1 · Florent · 2015 [cited by examiner]
US 20160216515A1 · Bouchier · 2016 [cited by examiner]
US 20170007350A1 · Popovic · 2017 [cited by examiner]
US 20170202633A1 · Liu · 2017 [cited by examiner]
US 20170357332A1 · Balan et al. · 2017 [cited by applicant]
US 20170372640A1 · Lampotang · 2017 [cited by examiner]
US 20180116714A1 · Karpiel · 2018 [cited by examiner]
US 20190088019A1 · Prevrhal · 2019 [cited by examiner]
US 20190192232A1 · Altmann · 2019 [cited by examiner]
US 20190290247A1 · Popovic · 2019 [cited by examiner]
US 20190374279A1 · Weitzner · 2019 [cited by examiner]
US 20200179039A1 · Yanuma · 2020 [cited by examiner]
US 20210100528A1 · Dayton · 2021 [cited by examiner]
US 20210137350A1 · Inglis · 2021 [cited by examiner]
US 20210169330A1 · Komp · 2021 [cited by examiner]
US 20210196399A1 · Ayvali · 2021 [cited by examiner]
US 20220104896A1 · Shelton, IV · 2022 [cited by examiner]
US 20220110504A1 · Inglis · 2022 [cited by examiner]
WO WO2016028828A1 · 2016 [cited by applicant]
Liu et al., “Monitoring with Head-Mounted Displays: Performance and Safety in a Full-Scale Simulator and Part-Task Trainer”, 2009 (Year: 2009). [cited by examiner]
Katsinelos et al., “Comparison of Three Types of Precut Technique to Achieve Common Bile Duct Cannulation: A Retrospective Analysis of 274 Cases”, 2011 (Year: 2011). [cited by examiner]
Shimatani et al., “Tips for double balloon enteroscopy in patients with Roux-en-Y reconstruction and modified Child surgery”, 2013 (Year: 2013). [cited by examiner]
Nguyen et al., “Markerless Tracking for Augmented Reality for Image-Guided Endoscopic Retrograde Cholangiopancreatography”, 2013 (Year: 2013). [cited by examiner]
Onda et al., “Augmented-reality based navigation surgery for hepatobiliary diseases”, 2014 (Year: 2014). [cited by examiner]
Tang et al., “Augmented reality technology for preoperative planning and intraoperative navigation during hepatobiliary surgery: A review of current methods”, 2017 (Year: 2017). [cited by examiner]
Diana et al., “Prospective Evaluation of Precision Multimodal Gallbladder Surgery Navigation”, 2017 (Year: 2017). [cited by examiner]
Chu et al., “Registration and fusion quantification of augmented reality based nasal endoscopic surgery”, 2017 (Year: 2017). [cited by examiner]
Andrews et al., “Extended Reality in Medical Practice”, 2019 (Year: 2019). [cited by examiner]
Tang et al., “Boosting Healthcare Performance: The Convergence ofArtificial Intelligence with Gastroenterology”, 2019 (Year: 2019). [cited by examiner]
Davis et al., “Evaluation of real-time guidewire navigation using virtual endoscopic 4D fluoroscopy”, 2020 (Year: 2020). [cited by examiner]
Pöhlmann et al., “Evaluation of Kinect 3D Sensor for Healthcare Imaging”, J. Med. Biol. Eng. (2016) 36:857-870; 14 pp. total. [cited by applicant]
Jerbi et al., “Anatomical and morphometric study of gastrointestinal tract of donkey (Equus africanus asinus)”, J. Morphol. Sci., 2014, vol. 31, No. 1, p. 18-22; 5 pp. total. [cited by applicant]
Edmundowicz, Steven A, “Wire Guided Cannulation: Clinical Perspective, a Review of the Literature and Appraisal of the Technique”, Mar. 2013; 4 pp. total. [cited by applicant]