Planning and performing three-dimensional holographic interventional procedures with holographic guide
Embodiments of the present disclosure may include a method for planning and performing an interventional procedure on a patient, the method including steps of providing an augmented reality system, a tracked instrument, a first image acquisition system, a second image acquisition system, and a computer system with a processor and a memory, the tracked instrument having a plurality of sensors to provide a tracked instrument dataset, the first image acquisition system, and the computer system in communication with the augmented reality system, the tracked instrument, the first image acquisition system, and the second image acquisition system. Embodiments may also include acquiring, by the first image acquisition system, the first holographic image dataset from the patient. Embodiments may also include acquiring, by the second image acquisition system, the second holographic image dataset from the patient.
1 . A method for planning and performing an interventional procedure on a patient by a practitioner, the method comprising:
providing an augmented reality system, a tracked instrument, a first image acquisition system, a second image acquisition system, and a computer system with a processor and a memory, the tracked instrument having a plurality of sensors configured to provide a tracked instrument dataset, the computer system in communication with the augmented reality system, the tracked instrument, the first image acquisition system, and the second image acquisition system;
acquiring, by the first image acquisition system, a first holographic image dataset from the patient;
acquiring, by the second image acquisition system, a second holographic image dataset from the patient;
tracking, by the computer system, the tracked instrument using the plurality of sensors to provide the tracked instrument dataset;
registering, by the computer system, the first holographic image dataset, the second holographic image dataset, and the tracked instrument dataset with the patient;
rendering, by the augmented reality system, a first hologram from the first holographic image dataset, a second hologram from the second holographic image dataset, and a treatment zone hologram from the tracked instrument dataset;
adjusting, with the computer system, the treatment zone hologram;
stamping, by the practitioner, the treatment zone hologram to lock in a planned trajectory, depth, and predictive treatment zone;
rendering, by the augmented reality system, a guide hologram based on the stamped treatment zone hologram; and
performing, by the practitioner, the interventional procedure on the patient while viewing the patient, the first hologram, and the guide hologram with the augmented reality system, whereby the practitioner employs the guide hologram and the augmented reality system for placement of the tracked instrument during the interventional procedure.
2 . The method of claim 1 , wherein the adjusting, with the computer system, the treatment zone hologram and the rendering, by the augmented reality system, the guide hologram are performed until an entire treatment area is covered by the treatment zone hologram.
3 . The method of claim 1 , wherein the first image acquisition system is one of a magnetic resonance imaging (MRI) apparatus and a computerized tomography (CT) apparatus.
4 . The method of claim 2 , wherein the first holographic image dataset from the patient is one of a preoperative image and a perioperative feedback loop.
5 . The method of claim 1 , wherein the first holographic image dataset from the patient is preoperative.
6 . The method of claim 1 , wherein the second image acquisition system is an ultrasound apparatus.
7 . The method of claim 1 , wherein the method includes use of a plurality of tracked instruments having a plurality of treatment zone holograms.
8 . The method of claim 1 , further comprising a step of assessing the treatment zone hologram versus reality.
9 . The method of claim 1 , further comprising a step of using the augmented reality system to simultaneously display the first hologram and the treatment zone hologram.
10 . The method of claim 9 , wherein the guide hologram is displayed simultaneously with the first hologram and the treatment zone hologram.
11 . The method of claim 1 , further comprising a step of collecting, by the computer system, operating information.
12 . The method of claim 11 , wherein the operating information includes real-time navigation instructions.
13 . The method of claim 1 , further comprising a step of projecting, by the augmented reality system, operating information over the tracked instrument.
14 . The method of claim 1 , wherein the guide hologram includes at least one of a hub, a guide path, and a hoop.
15 . The method of claim 1 , further comprising a step of generating, by the augmented reality system, a feedback based on the first holographic image dataset, the second holographic image dataset, and the tracked instrument dataset.
16 . The method of claim 15 , wherein the feedback includes at least one of visual feedback, auditory feedback, and numerical holographic feedback.
17 . The method of claim 1 , further comprising recording, using the computer system, a member selected from a group consisting of: the first holographic image dataset, the second holographic image dataset, the tracked instrument dataset, the guide hologram, and combinations thereof.
18 . The method of claim 1 , wherein the guide hologram is within a treatment zone at a distance from the tracked instrument.
19 . A system for planning and performing an interventional procedure on a patient by a practitioner, comprising:
an augmented reality system;
a tracked instrument having a plurality of sensors;
a first image acquisition system configured to acquire a first holographic image dataset from the patient;
a second image acquisition system configured to acquire a second holographic image dataset from the patient;
a computer system with a processor and a memory, the computer system in communication with the augmented reality system, the tracked instrument, the first image acquisition system, and the second image acquisition system and configured by machine-readable instructions to:
track the tracked instrument using the plurality of sensors to provide a tracked instrument dataset;
register the first holographic image dataset, the second holographic image dataset, and the tracked instrument dataset with the patient;
receive stamping input from the practitioner via the augmented reality system to lock in a planned trajectory, depth, and predictive treatment zone of a treatment zone hologram and,
wherein the augmented reality system is configured to render a first hologram, a second hologram, a treatment zone hologram, and a guide hologram based on the stamping input from the practitioner for viewing by the practitioner, and
wherein the first hologram and the guide hologram are displayed to the practitioner during the interventional procedure on the patient using the augmented reality system, and the guide hologram and the augmented reality system are configured for use by the practitioner for placement of the tracked instrument during the interventional procedure.
20 . The method of claim 1 , wherein the stamping step includes using at least one of a voice command and a hand gesture to lock in the planned trajectory, depth, and predictive treatment zone, and wherein the method further comprises repeating the adjusting and stamping steps until an entire treatment area is covered by a plurality of stamped treatment zone holograms.