Virtual trajectory planning
Provided is a method that encompasses the determination of a virtual trajectory within a virtual representation of a patient's anatomy, wherein the trajectory is defined by a user's visual axis relative to the three-dimensional virtual representation of the patient's anatomy. The method includes acquiring image data which describes the three-dimensional virtual representation of the patient's body part, acquiring position data which describes a spatial position of a user's visual axis within a virtual-world co-ordinate system, determining visualization data based on the image data and the position data, and determining virtual-world trajectory data based on the position data.
1 . A computer-implemented method of determining a virtual trajectory with respect to a three-dimensional virtual representation of a patient's body part, the method comprising:
acquiring image data which describes the three-dimensional virtual representation of the patient's body part;
acquiring position data which describes a three-dimensional spatial position of a user's visual axis within a three-dimensional virtual-world co-ordinate system and derived from the spatial position of at least one of a head-mounted display device worn by a user, a head of the user, and/or eyes of the user;
determining visualization data based on the image data and the position data, which describes a three-dimensional perception of the three-dimensional virtual representation of the patient's body part as seen along the user's visual axis within the three-dimensional virtual-world co-ordinate system; and
determining virtual-world trajectory data based on the position data, which describes the three-dimensional spatial position of the virtual trajectory with respect to the three-dimensional virtual representation of the patient's body part within the three-dimensional virtual-world co-ordinate system, wherein the three-dimensional spatial position of the virtual trajectory is defined based on the three-dimensional spatial position of the user's visual axis within the virtual-world co-ordinate system and derived from the spatial position of at least one of the head-mounted display device worn by the user, the head of the user, and/or the eyes of the user, until a desired spatial position of the virtual trajectory with respect to the virtual representation of the patient's body part is established by a user input.
2 . The method according to claim 1 , wherein a real trajectory is determined with respect to a patient's actual body part, wherein the method further comprises:
determining transformation data which describes a transformation between the virtual-world co-ordinate system and a real-world co-ordinate system; and
determining real-world trajectory data based on the virtual-world trajectory data and the transformation data, which describes the spatial position of the real trajectory within the real-world co-ordinate system and with respect to the patient's actual body part registered with the virtual representation of the patient's body part.
3 . The method according to claim 2 , wherein the spatial position of the virtual representation of the patient's body part remains invariant with respect to the real-world co-ordinate system for at least a defined amount of time, and wherein the spatial position of the virtual trajectory within the virtual-world co-ordinate system is modified by changing the spatial position of the user's visual axis within the virtual-world co-ordinate system.
4 . The method according to claim 2 , wherein the spatial position of the virtual representation of the patient's body part is adjustable with respect to the real-world co-ordinate system, and wherein the spatial position of the virtual trajectory within the virtual-world co-ordinate system is modified by changing the spatial position of the virtual representation of the patient's body part with respect to the real-world co-ordinate system.
5 . The method according to claim 2 , wherein the virtual representation of the patient's body part within the virtual-world co-ordinate system is superimposed with the patient's actual body part within the real-world co-ordinate system for at least a defined amount of time.
6 . The method according to claim 1 , wherein the image described by the visualization data is shown to the user via a head-mounted display-device.
7 . The method according to claim 6 , wherein the visual axis is derived from an optical axis of the head-mounted display-device or an optical axis of at least one eye of the user.
8 . The method according to claim 1 , wherein the visualization data describes the image of the virtual representation of the patient's body part as seen along the user's visual axis from at least one of:
from outside the virtual representation of the patient's body part; and/or
from inside the virtual representation of the patient's body part.
9 . The method according to claim 8 , wherein the visualization data describes the image of the virtual representation of the patient's body part as seen along the user's visual axis from inside the virtual representation of the patient's body part, wherein the virtual representation of the patient's body part is enlarged to surround a user's position within the virtual-world co-ordinate system.
10 . The method according to claim 1 , wherein the image described by the visualization data includes a visualization of the virtual trajectory, and/or wherein the image described by the visualization data indicates whether a structure of the virtual representation of the patient's body part is hit by the virtual trajectory.
11 . The method according to claim 1 , wherein the image described by the visualization data indicates a distance between a structure of the virtual representation of the patient's body part and the virtual trajectory.
12 . The method according to claim 1 , wherein the image described by the visualization data indicates at least one of a forbidden zone for the virtual trajectory and a forbidden parameter of the virtual trajectory, each of which is defined by limitations of a later surgical or therapeutic procedure.
13 . The method according to claim 12 , wherein further parameters of the virtual trajectory are selected after a desired spatial position of the virtual trajectory with respect to the virtual representation of the patient's body part has been selected and/or wherein the further parameters are modified via gesture control.
14 . The method according to claim 13 , wherein the further parameters are modified via gesture control comprising head movements.
15 . The method according to claim 1 , wherein a desired spatial position and/or a desired parameter of at least one virtual trajectory is selected via gesture control comprising eye gesture control.
16 . The method according to claim 1 , wherein the spatial position of the virtual trajectory with respect to the virtual representation of the patient's body part within the virtual-world co-ordinate system is moved by:
changing a user's viewing direction with respect to the virtual representation of the patient's body part within the virtual-world co-ordinate system; and/or
changing a distance of a user's viewing position with respect to the virtual representation of the patient's body part within the virtual-world co-ordinate system.
17 . A non-transitory computer readable storage medium storing a program comprising program instructions that, when executed on at least one processor of a computer or loaded onto the at least one processor of the computer, causes the computer to perform a method of determining a virtual trajectory with respect to a three-dimensional virtual representation of a patient's body part, the method comprising:
acquiring image data which describes the three-dimensional virtual representation of the patient's body part;
acquiring position data which describes a three-dimensional spatial position of a user's visual axis within a three-dimensional virtual-world co-ordinate system and derived from the spatial position of at least one of a head-mounted display device worn by a user, a head of the user, and/or eyes of the user;
determining visualization data based on the image data and the position data, which describes a three-dimensional perception of the three-dimensional virtual representation of the patient's body part as seen along the user's visual axis within the three-dimensional virtual-world co-ordinate system; and
determining virtual-world trajectory data based on the position data, which describes the three-dimensional spatial position of the virtual trajectory with respect to the three-dimensional virtual representation of the patient's body part within the virtual-world co-ordinate system, wherein the three-dimensional spatial position of the virtual trajectory is defined based on the three-dimensional spatial position of the user's visual axis within the virtual-world co-ordinate system and derived from the spatial position of at least one of the head-mounted display device worn by the user, the head of the user, and/or the eyes of the user, until a desired spatial position of the virtual trajectory with respect to the virtual representation of the patient's body part is established by a user input.
18 . A medical system for determining a virtual trajectory with respect to a three-dimensional virtual representation of a patient's body part, comprising:
at least one computer configured to:
acquire image data which describes a three-dimensional virtual representation of a patient's body part;
acquire position data which describes a three-dimensional spatial position of a user's visual axis within a three-dimensional virtual-world co-ordinate system and derived from the spatial position of at least one of a head-mounted display device worn by a user, a head of the user, and/or eyes of the user;
determine visualization data based on the image data and the position data, which describes a three-dimensional perception of the three-dimensional virtual representation of the patient's body part as seen along the user's visual axis within the three-dimensional virtual-world co-ordinate system; and
determine virtual-world trajectory data based on the position data, which describes the three-dimensional spatial position of the virtual trajectory with respect to the three-dimensional virtual representation of the patient's body part within the three-dimensional virtual-world co-ordinate system, wherein the three-dimensional spatial position of the virtual trajectory is defined based on the three-dimensional spatial position of the user's visual axis within the virtual-world co-ordinate system and derived from the spatial position of at least one of the head-mounted display device worn by the user, the head of the user, and/or the eyes of the user, until a desired spatial position of the virtual trajectory with respect to the virtual representation of the patient's body part is established by a user input;
at least one electronic data storage device storing at least one of the trajectory data and image data; and
a display device displaying at least the visualization data to a user, wherein the at least one computer is operably coupled to
the at least one electronic data storage device for acquiring, from the at least one electronic data storage device, at least one of the trajectory data and the image data; and
the display device for issuing a control signal to the display device for controlling the operation of the display device based on at least one of the trajectory data and the image data.
19 . The medical system according to claim 18 , wherein a real trajectory is determined with respect to a patient's actual body part, wherein the at least one computer is further configured to:
determine transformation data which describes a transformation between the virtual-world co-ordinate system and a real-world co-ordinate system; and
determine real-world trajectory data based on the virtual-world trajectory data and the transformation data, which describes the spatial position of the real trajectory within the real-world co-ordinate system and with respect to the patient's actual body part registered with the virtual representation of the patient's body part.
20 . The medical system according to claim 18 , wherein the image described by the visualization data:
includes a visualization of the virtual trajectory and/or indicates whether a structure of the virtual representation of the patient's body part is hit by the virtual trajectory; and
wherein the visualization data describes the image of the virtual representation of the patient's body part as seen along the user's visual axis from at least one of:
from outside the virtual representation of the patient's body part; and/or
from inside the virtual representation of the patient's body part.