Generating images for at least two displays in image-guided surgery
View Patent ↗A system, in particular for image-guided surgery, comprising: at least two display devices; a position determinator for determining the relative position of the display devices; and an image generator for generating images, which are to be displayed by the display devices, in accordance with the determined relative position.
1. A system for use in an associated image-guided procedure, the system comprising:
a first display device;
a second display device different than the first display device;
a position determinator, the position determinator determining a relative position between the first and second display devices;
an image generator, the image generator being configured to generate a first image to be displayed by the first display device in accordance with the determined relative position between the first and second display devices, and the image generator being configured to generate a second image to be displayed by the second display device in accordance with the determined relative position between the first and second display devices;
a viewer detector, the viewer detector detecting an associated person viewing at least one of the first display device and/or the second display device; and
a viewing tracker, the viewing tracker determining as a viewed display device which of the first and/or second display devices is being viewed by the associated person,
wherein the image generator generates a particular first image for the viewed display device of the at least one of the first and/or second display devices in accordance with the relative position between the first and second display devices,
wherein the viewed display device of the at least one of the first and/or second display devices persistently displays at least some of the particular first image.
2. The system according to claim 1 , wherein:
the viewer detector comprises a camera detecting the associated person viewing the at least one of the first and/or second display devices;
the viewing tracker comprises an eye tracker determining as the viewed display device which of the first and/or second display devices is being viewed by the associated person by tracking an eye position of the associated person.
3. The system according to claim 2 , further comprising:
a first adjustable mounting supporting the first display device relative to an associated base, the first adjustable mounting comprising a plurality of first support elements mutually connected via a set of first adjustable joints;
a second adjustable mounting supporting the second display device relative to the associated base, the second adjustable mounting comprising a plurality of second support elements mutually connected via a set of second adjustable joints; and
at least one sensor operably connected with the position determinator, the at least one sensor detecting a state of at least one of the first and/or second adjustable mountings,
wherein the position determinator determines the relative position between the first and second display devices in accordance with the state of the at least one of the first and/or second adjustable mountings detected by the at least one sensor.
4. The system according to claim 3 , wherein:
the at least one sensor detects the state of the at least one of the first and/or second adjustable mountings by detecting a state of one or more joints of the set of first adjustable joints and/or the one or more of the set of second adjustable joints,
the position determinator determines the relative position between the first and second display devices in accordance with the state of the one or more joints of the set of first adjustable joints and/or the one or more of the set of second adjustable joints detected by the at least one sensor.
5. The system according to claim 2 , further comprising:
a marker device operatively coupled with at least one of the first and/or second display devices,
wherein the position determinator is configured to determine the relative position between the first and second display devices in accordance with a determined position of the marker device.
6. The system according to claim 2 , further comprising:
at least one camera adapted to observe at least one of the first and/or second display devices, and to generate an output image of the observed at least one of the first and/or second display devices,
wherein the position determinator is configured to determine the relative position between the first and second display devices in accordance with the output image of the observed at least one of the first and/or second display devices generated by the at least one camera.
7. The system according to claim 2 , further comprising:
a camera operatively coupled with at least one of the first and/or second display devices, the camera being adapted to generate an output image of an observed field of view of the camera,
wherein the position determinator determines the relative position between the first and second display devices in accordance with the output image of the observed field of view of the camera generated by the camera.
8. The system according to claim 2 , further comprising:
a first adjustable mounting supporting the first display device relative to an associated base, the first adjustable mounting comprising a plurality of first support elements mutually connected via one or more first adjustable joints;
a second adjustable mounting supporting the second display device relative to the associated base, the second adjustable mounting comprising a plurality of second support elements mutually connected via one or more second adjustable joints; and
at least one actuator, the at least one actuator being operative to adjust a state and/or position of at least one of the first and/or second adjustable joints.
9. The system according to claim 2 , wherein:
the viewer detector identifies the associated person who is viewing the at least one of the first and/or second display devices as an identified viewer; and
the image generator generates the particular first image for the viewed display device of the at least one of the first and/or second display devices in accordance with:
the relative position between the first and second display devices; and
the identified viewer.
10. A method of generating images developed in an associated image-guided procedure for first and second display devices of an associated system, the method comprising:
determining by a position determinator of the associated system a relative position between the first and second display devices;
generating by an image generator of the associated system a first image to be displayed by the first display device of the associated system in accordance with the determined relative position between the first and second display devices, and a second image to be displayed by the second display device of the associated system in accordance with the determined relative position between the first and second display devices;
detecting by a viewer detector an associated person viewing at least one of the first display device and/or the second display device;
determining by a viewing tracker as a viewed display device which of the first and/or second display devices is being viewed by the associated person;
generating by the image generator a particular first image for the viewed display device of the at least one of the first and/or second display devices in accordance with the relative position between the first and second display devices; and
persistently displaying at least some of the particular first image by the viewed display device of the at least one of the first and/or second display devices.
11. The method according to claim 10 , further comprising:
using a camera of the viewer detector to detect the associated person viewing the at least one of the first and/or second display devices; and
using an eye tracker of the viewing tracker to determine as the viewed display device which of the first and/or second display devices is being viewed by the associated person by tracking an eye position of the associated person using the eye tracker.
12. The method according to claim 11 , further comprising:
detecting, by at least one sensor operably connected with the position determinator, a state of a first adjustable mounting supporting the first display device relative to an associated base and/or a second adjustable mounting supporting the second display device relative to the associated base, the first adjustable mounting comprising a plurality of first support elements mutually connected via a set of first adjustable joints, the second adjustable mounting comprising a plurality of second support elements mutually connected via a set of second adjustable joints; and
determining by the position determinator the relative position between the first and second display devices in accordance with the state of the at least one of the first and/or second adjustable mountings detected by the at least one sensor.
13. The method according to claim 12 , further comprising:
detecting, by the at least one sensor, the state of the at least one of the first and/or second adjustable mountings by detecting a state of one or more joints of the set of first adjustable joints and/or the one or more of the set of second adjustable joints; and
determining, by the position determinator, the relative position between the first and second display devices in accordance with the state of the one or more joints of the set of first adjustable joints and/or the one or more of the set of second adjustable joints detected by the at least one sensor.
14. The method according to claim 11 , further comprising:
determining by the position determinator the relative position between the first and second display devices in accordance with a determined position of a marker device operatively coupled with at least one of the first and/or second display devices.
15. The method according to claim 11 , further comprising:
observing by at least one associated camera at least one of the first and/or second display devices;
generating by the at least one associated camera an output image of the observed at least one of the first and/or second display devices; and
determining by the position determinator the relative position between the first and second display devices in accordance with the output image generated by the associated at least one camera.
16. The method according to claim 11 , further comprising:
observing by an associated camera a field of view of the associated camera;
generating by the associated camera an output image of the field of view of the associated camera; and
determining by the position determinator the relative position between the first and second display devices in accordance with the output image generated by the associated camera.
17. The method according to claim 11 , further comprising:
generating a drive signal, the drive signal driving at least one associated actuator for adjusting a state of at least one of first adjustable joints mutually connecting a plurality of first support elements comprising a first adjustable mounting supporting the first display device relative to an associated base and/or second adjustable joints mutually connecting a plurality of second support elements comprising a second adjustable mounting supporting the second display device relative to the associated base.
18. The method according to claim 11 , further comprising:
identifying by the viewer detector the associated person who is viewing the at least one of the first and/or second display devices as an identified viewer; and
generating by the image generator the particular first image for the viewed display device of the at least one of the first and/or second display devices in accordance with:
the relative position between the first and second display devices; and
the identified viewer.
19. A non-transitory machine readable storage medium having instructions stored thereon that when executed cause an associated system including first and second display devices to perform in an associated image-guided procedure steps comprising:
determining by a position determinator of the associated system a relative position between the first and second display devices;
generating by an image generator of the associated system a first image to be displayed by the first display device of the associated system in accordance with the determined relative position between the first and second display devices, and a second image to be displayed by the second display device of the associated system in accordance with the determined relative position between the first and second display devices;
detecting by a viewer detector an associated person viewing at least one of the first display device and/or the second display device;
determining by a viewing tracker as a viewed display device which of the first and/or second display devices is being viewed by the associated person;
generating by the image generator a particular first image for the viewed display device of the at least one of the first and/or second display devices in accordance with the relative position between the first and second display devices; and
persistently displaying at least some of the particular first image by the viewed display device of the at least one of the first and/or second display devices.
20. The storage medium according to claim 19 having the instructions stored thereon that when executed cause the associated system to perform further steps comprising:
using a camera of the viewer detector to detect the associated person viewing the at least one of the first and/or second display devices; and
using an eye tracker of the viewing tracker to determine as the viewed display device which of the first and/or second display devices is being viewed by the associated person by tracking an eye position of the associated person using the eye tracker.
21. The storage medium according to claim 20 , having the instructions stored thereon that when executed cause the associated system to perform further steps comprising:
detecting, by at least one sensor operably connected with the position determinator, a state of a first adjustable mounting supporting the first display device relative to an associated base and/or a second adjustable mounting supporting the second display device relative to the associated base, the first adjustable mounting comprising a plurality of first support elements mutually connected via a set of first adjustable joints, the second adjustable mounting comprising a plurality of second support elements mutually connected via a set of second adjustable joints; and
determining by the position determinator the relative position between the first and second display devices in accordance with the state of the at least one of the first and/or second adjustable mountings detected by the at least one sensor.
22. The storage medium according to claim 21 , having the instructions stored thereon that when executed cause the associated system to perform further steps comprising:
detecting, by the at least one sensor, the state of the at least one of the first and/or second adjustable mountings by detecting a state of one or more joints of the set of first adjustable joints and/or the one or more of the set of second adjustable joints; and
determining, by the position determinator, the relative position between the first and second display devices in accordance with the state of the one or more joints of the set of first adjustable joints and/or the one or more of the set of second adjustable joints detected by the at least one sensor.
23. The storage medium according to claim 20 , having the instructions stored thereon that when executed cause the associated system to perform further steps comprising:
determining by the position determinator the relative position between the first and second display devices in accordance with a determined position of a marker device operatively coupled with at least one of the first and/or second display devices.
24. The storage medium according to claim 20 , having the instructions stored thereon that when executed cause the associated system to perform further steps comprising:
observing by at least one associated camera at least one of the first and/or second display devices;
generating by the at least one associated camera an output image of the observed at least one of the first and/or second display devices; and
determining by the position determinator the relative position between the first and second display devices in accordance with the output image generated by the associated at least one camera.
25. The storage medium according to claim 20 , having the instructions stored thereon that when executed cause the associated system to perform further steps comprising:
observing by an associated camera a field of view of the associated camera;
generating by the associated camera an output image of the field of view of the associated camera; and
determining by the position determinator the relative position between the first and second display devices in accordance with the output image generated by the associated camera.
26. The storage medium according to claim 20 , having the instructions stored thereon that when executed cause the associated system to perform further steps comprising:
generating a drive signal, the drive signal driving at least one associated actuator for adjusting a state of at least one of first adjustable joints mutually connecting a plurality of first support elements comprising a first adjustable mounting supporting the first display device relative to an associated base and/or second adjustable joints mutually connecting a plurality of second support elements comprising a second adjustable mounting supporting the second display device relative to the associated base.
27. The storage medium according to claim 20 , having the instructions stored thereon that when executed cause the associated system to perform further steps comprising:
identifying by the viewer detector the associated person who is viewing the at least one of the first and/or second display devices as an identified viewer; and
generating by the image generator the particular first image for the viewed display device of the at least one of the first and/or second display devices in accordance with:
the relative position between the first and second display devices; and
the identified viewer.