Exchange of data between an external data source and an integrated medical data display system

1 . A method for exchanging data between an external data source for annotations and an integrated medical data display system, the method comprising:

determining information displayed on a screen of the integrated medical data display system by capturing the screen, the information displayed on the screen including formal information about a format of displayed data, the formal information including a screen layout with a delimited screen portion including an image segment and a second screen portion not containing the image segment;

selecting data from the external data source assigned to the information displayed on the screen;

displaying the selected data as complementary to the information displayed on the screen by showing the selected data in an external window opened next to the integrated medical data display system; and

displaying an external application window within the second screen portion.

2 . The method according to claim 1 , wherein the information displayed on the screen further comprises at least one of:

content of medical data, or

non-image information including patient metadata or examination metadata.

3 . The method according to claim 1 , wherein

the external data source includes results, and

the selecting data from the external data source includes searching the external data source for results corresponding to the information displayed on the screen, wherein the selected data includes the results.

4 . The method according to claim 1 , wherein the determining the information displayed on the screen of the integrated medical data display system comprises:

detecting a reference frame based on the information displayed on the screen, and

detecting annotations in the reference frame based on the information displayed on the screen.

5 . The method of claim 1 , wherein the format of the displayed data includes information about a shape and size of displayed structures including windows and sub-windows of the screen.

6 . The method according to claim 1 , wherein the determining the information displayed on the screen comprises:

determining corner locations or borders of displayed viewports on the screen based on the screen layout.

7 . The method according to claim 6 , wherein the determining the information displayed on the screen comprises:

using a metric to determine at least one screen portion as a monitor segment of least-interest based on the information displayed on the screen, and

displaying a further external application window within the monitor segment of least-interest.

8 . The method according to claim 7 , wherein in case a plurality of monitor segments of least-interest are determined, a position for the further external application window is chosen at least one of

based on a best size fit between the external application window and the plurality of monitor segments of least-interest,

based on an aspect ratio fit between the external application window and the plurality of monitor segments of least-interest, or

overlapping the external application window with images of the integrated medical data display system, in response to no monitor segment of least interest that fits the external application window being found.

9 . The method according to claim 1 , wherein

the formal information further includes at least one of:

the screen layout including at least one further screen portion, each further screen portion including at least one of a viewport or a further delimited screen portion including the image segment, or

user input to the integrated medical data display system.

10 . The method according to claim 9 , wherein the determining the information displayed on the screen further comprises:

determining at least one active viewport using mouse activity tracked using screen capture.

11 . The method according to claim 9 , wherein the determining the information displayed on the screen comprises at least one of:

using a tool capable of screen tracking to capture screen content,

detecting which study or patient is currently displayed on the screen,

classifying, based on the screen layout, which screen portion corresponds to which image segment, and identifying, for each classified image segment, which image a user is at least one of currently visualizing or active on, or

identifying a window level the user is working with for a particular slice.

12 . The method according to claim 11 , wherein the determining the information displayed on the screen comprises:

determining corner locations or borders of displayed viewports on the screen based on the screen layout.

13 . The method according to claim 11 , wherein the determining the information displayed on the screen further comprises:

determining at least one active viewport using mouse activity tracked using screen capture.

14 . The method according to claim 11 , wherein

the external data source includes results, and

the selecting data from the external data source includes searching the external data source for results corresponding to the information displayed on the screen, wherein the selected data includes the results.

15 . The method according to claim 14 , wherein the displaying the selected data comprises:

generating an overlay on the information displayed on the screen of the integrated medical data display system, wherein

the overlay includes annotations corresponding to the information displayed on the screen, and

the overlay is registered with the information displayed on the screen.

16 . A data exchange device for exchanging data between an external data source for annotations and an integrated medical data display system, the data exchange device comprising:

a determination unit configured to determine information displayed on a screen of the integrated medical data display system by capturing the screen, the information displayed on the screen including formal information about a format of displayed data, the formal information including a screen layout with a delimited screen portion including an image segment and a second screen portion not containing the image segment, wherein the second screen portion includes an external application window; and

a selection unit configured to select data from the external data source assigned to the information displayed on the screen and display the selected data as complementary to the information displayed on the screen, by showing the selected data in a particular external window that is opened next to the integrated medical data display system.

17 . A non-transitory computer-readable medium comprising instructions that, when executed by a computer, cause the computer to carry out the method of claim 1 .

18 . A data exchange device for exchanging data between an external data source for annotations and an integrated medical data display system, the data exchange device comprising:

a memory storing computer executable instructions; and

at least one processor configured to execute the computer executable instructions to cause the data exchange device to

determine information displayed on a screen of the integrated medical data display system by capturing the screen, the information displayed on the screen including formal information about a format of displayed data, the formal information including a screen layout with a delimited screen portion including an image segment and a second screen portion not containing the image segment,

select data from the external data source assigned to the information displayed on the screen,

display the selected data as complementary to the information displayed on the screen by showing the selected data in an external window opened next to the integrated medical data display system, and

display an external application window within the second screen portion.