Heartbeat based selection of images for cardiovascular model
To create a 3D model of part of a cardiovascular system, two 2D images taken of different orientations of the cardiovascular system may be combined. The 2D images originate from video streams taken at different points in time, which comprise frames showing a beating heart, and thus a moving cardiovascular system. Because of this movement, not just any random set of two 2D images may result in useable 3D model. To select a proper set of two 2D images, a method is provided wherein said selection is based on cardiac cycle data. The cardiac cycle data may comprise heart activity data as a function of time and timing data on cycle events. These cycle events may be repetitive, as the same events occur with every heartbeat. The selected frames are preferably selected at, or approximately at, similar events.
1. A method of building a 3D model of at least a part of a cardiovascular system of a person, comprising:
receiving a first video stream comprising first frames comprising image data on the cardiovascular system over a first period of time;
receiving a second video stream comprising second frames comprising image data on the cardiovascular system over a second period of time, wherein the first period of time is in disjunction from the second period of time;
obtaining cardiac cycle data related to the cardiovascular system from the first period of time and the second period of time;
selecting at least one of the first frames and at least one of the second frames based on the cardiac cycle data; and
building the 3D model of part of the cardiovascular system using the selected frames,
wherein the cardiac cycle data comprises timing data on a first cycle event within the first period of time and timing data on a second cycle event within the second period of time, and the selecting of the at least one of the first frames is based on the timing data of the first cycle event and the selecting of the at least one of the second frames is based on the timing data of the second cycle event.
2. The method according to claim 1 , wherein the cardiac cycle data comprises heart activity data as a function of time.
3. The method according to claim 1 , wherein the second cycle event is equivalent to the first cycle event.
4. The method according to claim 1 , further comprising:
selecting, from the first frames, based on the first cycle event, multiple first selected frames related to the first cycle event;
selecting, from the second frames, based on the second cycle event, multiple second selected frames related to the second cycle event;
identifying, in at least a first of the first selected frames and the second selected frames, a first image region depicting a cardiovascular structure;
comparing, in the first of the first selected frames and the second selected frames, at least one characteristic of the first image region;
based on the comparing, selecting from the first of the first selected frames and the second selected frames, a first model frame; and
building the 3D model using the first model frame.
5. The method according to claim 4 , further comprising:
identifying, in a second of the first selected frames and the second selected frames, a second image region depicting the cardiovascular structure;
comparing, in the second of the first selected frames and the second selected frames, at least one characteristic of the second image region;
based on the comparing, selecting from the second of the first selected frames and the second selected frames, a second model frame; and
building the 3D model using the first model frame and the second model frame.
6. The method according to claim 4 , further comprising determining, in the first of the first selected frames and the second selected frames, a length of the cardiovascular structure;
wherein the at least one characteristic of the first image region is the length of the cardiovascular structure.
7. The method according to claim 4 , further comprising determining, within the first image region depicting the cardiovascular structure, in at least the first of the first selected frames and the second selected frames, a brightness level of the first image region or a sub-region thereof;
wherein the at least one characteristic of the first image region is the brightness level.
8. The method according to claim 4 , wherein, in the step of comparing, the first of the first selected frames and the second selected frames are subsequent in time.
9. The method according to claim 8 , further comprising determining, within the first image region depicting the cardiovascular structure, in the first of the first selected frames and the second selected frames, a brightness level of the first image region or a sub-region thereof;
wherein:
the at least one characteristic of the first image region is the brightness level; and
the first model frame is selected based on having the brightness level that is lower than that of the first image region or sub-region thereof in a subsequent frame.
10. The method according to claim 8 , further comprising:
determining, in the first of the first selected frames and the second selected frames, a length of the cardiovascular structure; and
determining, within the first image region depicting the cardiovascular structure, in the first of the first selected frames and the second selected frames, a brightness level of the first image region or a sub-region thereof;
wherein:
one of the at least one characteristic of the first image region is the length of the cardiovascular structure; and
another of the at least one characteristic of the first image region is the brightness level; and
the first model frame is selected based on having the longest length and the brightness level that is lower than that of the first image region or sub-region thereof in a subsequent frame.
11. The method according to claim 1 , wherein the first cycle event and the second cycle event correspond to an end of diastole.
12. The method according to claim 1 , further comprising receiving electrocardiography data from the first period of time and the second period of time, wherein the cardiac cycle data is obtained from the electrocardiography data.
13. The method according to claim 1 , further comprising receiving photoplethysmography data from the first period of time and the second period of time, wherein the cardiac cycle data is obtained from the photoplethysmography data.
14. The method according to claim 1 , wherein the image data over the first period of time and the second period of time are obtained by a single medical imaging device.
15. The method according to claim 1 , carried out by a computer.
16. A non-transitory computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 1 .
17. A method of building a 3D model of at least a part of a cardiovascular system of a person, comprising:
receiving a first video stream comprising first frames comprising image data on the cardiovascular system over a first period of time;
receiving a second video stream comprising second frames comprising image data on the cardiovascular system over a second period of time, wherein the first period of time and the second period of time partially or fully overlap;
obtaining cardiac cycle data related to the cardiovascular system from the first period of time and the second period of time;
selecting at least one of the first frames and at least one of the second frames based on the cardiac cycle data; and
building the 3D model of part of the cardiovascular system using the selected frames,
wherein the cardiac cycle data comprises timing data on a first cycle event within the first period of time and timing data on a second cycle event within the second period of time, and the selecting of the at least one of the first frames is based on the timing data of the first cycle event and the selecting of the at least one of the second frames is based on the timing data of the second cycle event.
18. The method according to claim 17 , wherein the cardiac cycle data comprises heart activity data as a function of time.
19. The method according to claim 17 , wherein the second cycle event is equivalent to the first cycle event.
20. The method according to claim 17 , further comprising:
selecting, from the first frames, based on the first cycle event, multiple first selected frames related to the first cycle event;
selecting, from the second frames, based on the second cycle event, multiple second selected frames related to the second cycle event;
identifying, in at least a first of the first selected frames and the second selected frames, a first image region depicting a cardiovascular structure;
comparing, in the first of the first selected frames and the second selected frames, at least one characteristic of the first image region;
based on the comparing, selecting from the first of the first selected frames and the second selected frames, a first model frame; and
building the 3D model using the first model frame.