Method and system for displaying holographic images within a real object
A system for displaying a holographic image of an object behind a real object surface, including a computing unit for computing data for displaying a three-dimensional image of an object, a location measurement unit for measuring a location of a surface of a real object, a display for displaying the three dimensional image of the object, wherein the computing unit is adapted to compute data to display the three-dimensional image of the object at least partly behind the surface of the real object. Related apparatus and methods are also described.
1 . A method for producing an image comprising visual representations of a real internal organ of a patient and a real medical tool which are at least partially hidden from view in the real-world by the patient's body, the image depicting the internal organ and the medical tool as visible, and further as located in space at the same real-world positions as the respective real internal organ and the real medical tool relative to a body of the patient, the method comprising:
(a) obtaining, by a circuitry, a dataset comprising a three-dimensional (3D) location and orientation in space of the following:
i. an imaging device relative to a display;
ii. at least a portion of an internal organ of a patient
(b) tracking, by the circuitry, a location and orientation in space of the medical tool;
(c) generating, by the circuitry, locations of the internal organ and the medical tool relative to the display, based on the dataset and the tracked location of the medical tool;
(d) generating the visual representations of the at least a portion of the internal organ and the portion of the medical tool that are hidden from view;
(e) producing an image which includes the visual representations positioned based on the generated locations; and
(f) displaying, on the display, the image such that the visual representations appear in co-registration with their respective real-world internal organ and medical tool;
wherein the displaying comprises displaying the visual representation of the portion of the medical tool that is hidden from view as an extension of a portion of the real medical tool that is not hidden;
wherein the displaying comprises displaying with depth cues comprising eye focus accommodation corresponding to: a first depth within the real internal organ, and a second depth within the real medical tool.
2 . A method according to claim 1 , wherein obtaining the dataset comprises measuring the 3D location in space of the imaging device relative to the display by a measuring unit comprised in the display.
3 . A method according to claim 2 , wherein measuring the 3D location in space of the imaging device relative to the display comprises detecting a registration mark on the imaging device.
4 . A method according to claim 1 , wherein the image which displays the at least a portion of the internal organ and the portion of the medical tool is a 3D image.
5 . A method according to claim 1 in which the displaying the image comprises displaying the image at least partly behind a surface of the body of the patient.
6 . A method according to claim 1 , wherein the displaying the image comprises providing a viewer of the image with a plurality of eye convergence depth cues.
7 . A method according to claim 1 , wherein the displaying the image comprises providing a viewer of the image with a plurality of eye focus depth cues.
8 . A method according to claim 1 , wherein the displaying the image comprises providing a viewer of the image with a plurality of both eye convergence and eye focus depth cues.
9 . A method according to claim 1 wherein the display is configured to display a plurality of different points in the image as focused at a plurality of respective different distances from the display.
10 . A method according to claim 1 , wherein the imaging device comprises an ultrasound imaging device.
11 . A method according to claim 1 , wherein the displaying the image comprises displaying using a mobile display.
12 . A method according to claim 1 , wherein the tracking is in real-time, while displaying the image which displays the at least a portion of the internal organ and the portion of the medical tool.
13 . A method according to claim 1 , wherein the dataset is obtained in real-time, while displaying the image which displays the at least a portion of the internal organ and the portion of the medical tool.
14 . A method according to claim 1 , wherein:
the dataset is obtained in real-time, while displaying the image which displays the at least a portion of the internal organ and the portion of the medical tool, and
the tracking is in real-time, while displaying the image which displays the portion of the internal organ and the portion of the medical tool.
15 . A method according to claim 1 , wherein generating the visual representations comprises aligning the visual representations to the respective real-world internal organ and medical tool including location, orientation, size and scale.
16 . A system for producing an image comprising visual representations of a real internal organ of a patient and a real medical tool which are at least partially hidden from view in the real-world by the patient's body, the image depicting the internal organ and the medical tool as visible, and further as located in space at the same real-world positions as the respective real internal organ and the real medical tool, the system comprising:
a. a display for displaying an image;
b. a circuitry adapted to:
i. obtain a three-dimensional (3D) location in space of an imaging device relative to a display, and at least a portion of an internal organ of a patient;
ii. track a location and orientation in space of a medical tool;
iii. generate locations of the internal organ and the medical tool relative to the display, based on the dataset and the tracked location of the medical tool;
iv. generate the visual representations of the at least a portion of the internal organ and the portion of the medical tool that are hidden from view; and
v. produce the image including the visual representations positioned based on the generated locations;
vi. providing instruction to the display to display the image: such that the visual representations appear in co-registration with their respective real-world internal organ and medical tool; the visual representation of the portion of the medical tool that is hidden from view appears as an extension of a portion of the real medical tool that is not hidden; and
with depth cues comprising eye focus accommodation corresponding to: a first depth within the real internal organ, and a second depth within the real medical tool.
17 . A system according to claim 16 , wherein the circuitry is adapted to obtain the 3D location in space of the imaging device relative to the display by measuring the 3D location in space of the feature relative to the display by a measuring unit comprised in the display.
18 . A system according to claim 16 in which the display is a 3D display.
19 . A system according to claim 16 , wherein the imaging device comprises an ultrasound imaging device.
20 . A system according to claim 16 , wherein the display comprises a display selected from a group consisting of:
a Head Mounted Display (HMD);
a mobile display; and
a display on an adjustable arm.