Granted Patent B2

US 12,648,750 · App. 18/663,682 · Granted Jun 9, 2026

Inventors: Nastasja U Robaina (Coconut Grove, FL); Praveen Babu Jd (Plantation, FL); David Charles Lundmark (Los Altos, CA); Alexander Ilic (Zurich, CH)

Assignee: Magic Leap, Inc.

A61B6/547A61B5/06A61B5/065A61B5/066A61B6/022A61B6/032A61B6/464A61B6/466A61B6/5205G06T7/0012G06T15/00A61B2560/066A61B2562/0219

Abstract

Claims (70)

1 . A method of viewing a patient comprising:

inserting a tip of a catheter into a body of a patient;

detecting movement of the tip with a tip tracking device;

receiving a measurement based on movement detected by the tip tracking device;

determining a position of the tip based on the measurement;

storing the position of the tip;

determining left and right image data sets based on the position of the tip;

generating light in a pattern representative of the position of the tip using left and right projectors projecting the left and right image data sets respectively as light; and

guiding the light to retinas of left and right eyes of a viewer so that the viewer sees position of the tip, the left and right image data sets differing from one another to give the viewer a perception of a three-dimensional rendering.

2 . The method of claim 1 , further comprising:

mounting a head-mountable frame to a head of a viewer, the light wave guides being secured to the head-mountable frame.

3 . The method of claim 2 , wherein the light wave guides are transparent light wave guides positioned between the eyes and the external surface of the body.

4 . The method of claim 2 , further comprising:

detecting movement of the head-mountable frame;

calculating a placement value based on the movement detected; and

modifying a position of the body part within a view of the eye based on the placement value.

5 . The method of claim 4 , wherein the movement is detected with a motion sensor of a head unit inertial measurement unit (IMU) mounted to the head-mountable frame.

6 . The method of claim 4 , wherein the movement is detected with a head unit camera mounted to the head-mountable frame, the head unit camera detecting movement of the head-mountable frame by taking images of objects within view of the head unit camera.

7 . The method of claim 6 , further comprising:

analyzing the images to detect a pose position of the head-mountable frame.

8 . The method of claim 1 , further comprising:

activating a transmitter to generate a forward wave at a body part within the body;

detecting, with a receiver, a return wave from the body part, the return wave from the body part being in response to the forward wave created by the transmitter;

receiving raw data of the return wave detected by the receiver;

storing the raw data in a data store;

processing the raw data of the return wave to create image data representing an image;

storing the image data in the data store;

receiving the image data from the data store; and

combining the position of the tip with the image data, the pattern of light created including the pattern representative of the image data and the position of the tip.

9 . The method of claim 8 , wherein the transmitter is an x-ray transmitter secured to a rotor and transmitting an x-ray wave and the receiver is an x-ray detector secured to the rotor to detect an x-ray wave, movement of the platform relative to a base allowing for movement of the patient relative to a plane extending from the x-ray transmitter to the x-ray detector.

10 . The method of claim 1 , further comprising:

storing a past path of the tip in the data store; and

displaying the past path of the tip together with position of the tip.

11 . The method of claim 1 further comprising:

generating a three-dimensional mesh around the past path of the tip;

storing the mesh in the data store, the catheter display integrator; and

displaying the mesh together with position of the tip.

12 . The method of claim 11 , further comprising:

capturing video data with a catheter camera in the tip;

receiving the video data, the catheter display integrator displaying a live video based on the video data; and

displaying a live video based on the video data.

13 . The method of claim 1 , the set of instructions including:

calculating a future path of the tip based the position of the tip; and

displaying the future path.

14 . The method of claim 13 , wherein the future path is calculated based on the movement detected by the tip tracking device.

15 . The method of claim 14 , wherein the movement detected by the tip tracking device is a first amount of movement from a first position to a second position though a first angle in a selected direction and the future path is a second amount of movement from the second position to a third position through a second angle in the selected direction.

16 . The method of claim 1 , further comprising:

performing an invasive procedure with an invasive surgical tool on the tip.

17 . The method of claim 1 , further comprising:

capturing images with a camera on the tip; and

displaying images captures by the camera to an operator.

18 . The method of claim 17 , further comprising:

displaying the three-dimensional rendering and the images captured by the camera to a first operator.

19 . The method of claim 17 , further comprising:

displaying the three-dimensional rendering to a first operator; and

displaying the images captured by the camera to a second operator.

20 . The method of claim 1 , further comprising:

displaying the three-dimensional rendering to a first operator;

collecting data without the catheter; and

display the data collected without the catheter to a second operator.

21 . The method of claim 1 , further comprising:

determining operational load on the tip; and

warning an operator if a load on the tip exceeds a predetermined limit.

22 . The method of claim 21 , wherein the warning is one of a visual warning, an audio warning and a haptic warning.

23 . The method of claim 1 , further comprising:

observing motion artifacts; and

make (making) positional adjustments to the tip based on the motion artifacts.

24 . The method of claim 23 , wherein the motion detection system collects feedback control images of patient activity at three separate times at times T1 to T2 to T3, and the images are analyzed at time T4 to determine a change in patient position and an adjustment to the control input based on the change in patient position observed over the times T1 to T3 is made at time T5.

25 . The method of claim 24 , wherein:

IF( T 5− T 3= T 3− T 2)∩( y 3− y 2)<( y 2− y 1), then correction at T 5<( y 3− y 2).

Assignments (2)

Continuity (4)

References Cited (400)