Combining angiographic information with fluoroscopic images
According to the present invention, a plurality of image pairs, each consisting of a fluoroscopic image and an angiographic image taken from the same position and the same viewing direction onto the patient, is acquired. A live fluoroscopic image is registered individually with each fluoroscopic image within the image pairs, such that the spatial relationship between the live fluoroscopic image and the fluoroscopic images, and thus with the angiographic images and the image pairs in general, becomes known. Angiographic information representing the vascular structure can then be taken from those parts of the angiographic images within the image pairs which overlap with the live fluoroscopic image and be overlayed over the live fluoroscopic image.
1 . A method of overlaying angiographic information over a fluoroscopic image of a patient, executed by one or more processors of a fluoroscopy system having an X-ray source, a detector, and a display, the method comprising:
acquiring, via the one or more processors, a plurality of image pair data, each image pair data representing an image pair and comprising fluoroscopic image data and angiographic image data representing a fluoroscopic image and an angiographic image, respectively, of a part of the patient, the fluoroscopic image and the angiographic image of an image pair being taken from the same position and the same viewing direction onto the patient,
calculating, via the one or more processors, a set of image pair registration data, each image pair registration data representing the spatial orientation between two neighbouring image pairs,
acquiring, via the one or more processors, live fluoroscopic image data representing a live fluoroscopic image of the patient,
registering, via the one or more processors, the live fluoroscopic image data individually with the fluoroscopic image data of each image pair, thus obtaining a plurality of distinct spatial orientations between the live fluoroscopic image and each corresponding image pair,
determining, via the one or more processors, angiographic overlay image data from the angiographic image data based on the plurality of distinct spatial orientations between the live fluoroscopic image and each corresponding image pair, and
overlaying the angiographic overlay image data over the live fluoroscopic image.
2 . The method of claim 1 , further comprising the step of displaying an overlay of the live fluoroscopic image with one or more of the fluoroscopic images of the image pairs separately.
3 . The method of claim 1 , wherein the step of calculating the set of image pair registration data involves matching only the fluoroscopic image data of image pairs, matching only the angiographic image data of image pairs or matching both the fluoroscopic image data and the angiographic image data of image pairs.
4 . The method of claim 3 , further comprising the step of outputting warning information if the spatial orientation obtained by matching of the fluoroscopic image data differs from the spatial orientation obtained by matching the angiographic image data by more than a predetermined threshold.
5 . The method of claim 1 , further comprising the step of calculating angiographic panoramic image data from the angiographic image data of the image pairs, wherein determining the angiographic overlay image data involves selecting a part of the angiographic panoramic image data.
6 . The method of claim 5 , wherein the step of determining the angiographic panoramic image data involves stitching the angiographic image data, wherein overlapping parts of neighbouring angiographic image data are combined as a weighted sum.
7 . The method of claim 6 , wherein weights for the angiographic image data increase from an edge of the angiographic image towards the center.
8 . The method of claim 1 , wherein, when the live fluoroscopic image overlaps more than one fluoroscopic image of the image pairs, the method further comprises the step of determining overlapped area data representing overlapped areas of the angiographic images corresponding to those areas of the fluoroscopic images which are overlapped by the live fluoroscopic image and the step of determining the angiographic overlay image data uses the angiographic image data in the overlapped areas.
9 . The method of claim 8 , wherein the step of determining the angiographic overlay image data involves using the angiographic image data only of the one angiographic image which has the largest overlapped area.
10 . The method of claim 8 , wherein the step of determining the angiographic overlay image data involves copying, from the angiographic image which has the largest overlapped area, all of the angiographic image data in its overlapped area and copying those parts of the angiographic image data of the overlapped areas of other angiographic images which do not overlap with said largest overlapped area.
11 . The method of claim 8 , wherein the step of determining the angiographic overlay image data involves copying the angiographic image data corresponding to those areas within overlapped areas of angiographic images which do not overlap with an overlapped area of another angiographic image and merging angiographic image data of areas where overlapped areas of different angiographic images overlap.
12 . The method of claim 1 , further comprising the step of displaying an overlay of two neighbouring fluoroscopic images or of two neighbouring angiographic images based on the corresponding image pair registration data.
13 . The method of claim 1 , further comprising the step of marking those areas of the overlay angiographic image data for which an overlay confidence value is outside a predetermined range.
14 . A non-transitory computer readable medium comprising instructions which, when executed, cause at least one processor to perform operations, the operations comprising:
acquire, via the one or more processors, a plurality of image pair data, each image pair data representing an image pair and comprising fluoroscopic image data and angiographic image data representing a fluoroscopic image and an angiographic image, respectively, of a part of the patient, the fluoroscopic image and the angiographic image of an image pair being taken from the same position and the same viewing direction onto the patient,
calculate, via the one or more processors, a set of image pair registration data, each image pair registration data representing the spatial orientation between two neighbouring image pairs,
acquire, via the one or more processors, live fluoroscopic image data representing a live fluoroscopic image of the patient,
register, via the one or more processors, the live fluoroscopic image data individually with the fluoroscopic image data of each image pair, thus obtaining a plurality of distinct spatial orientations between the live fluoroscopic image and each corresponding image pair,
determine, via the one or more processors, angiographic overlay image data from the angiographic image data based on the plurality of distinct spatial orientations between the live fluoroscopic image and each corresponding image pair, and
overlay the angiographic overlay image data over the live fluoroscopic image.
15 . A medical system, comprising:
at least one processor having associated memory, the associated memory having instructions which, when executed by the at least one processor, cause the at least one processor to:
acquire, via the one or more processors, a plurality of image pair data, each image pair data representing an image pair and comprising fluoroscopic image data and angiographic image data representing a fluoroscopic image and an angiographic image, respectively, of a part of the patient, the fluoroscopic image and the angiographic image of an image pair being taken from the same position and the same viewing direction onto the patient,
calculate, via the one or more processors, a set of image pair registration data, each image pair registration data representing the spatial orientation between two neighbouring image pairs,
acquire, via the one or more processors, live fluoroscopic image data representing a live fluoroscopic image of the patient,
register, via the one or more processors, the live fluoroscopic image data individually with the fluoroscopic image data of each image pair, thus obtaining a plurality of distinct spatial orientations between the live fluoroscopic image and each corresponding image pair,
determine, via the one or more processors, angiographic overlay image data from the angiographic image data based on the plurality of distinct spatial orientations between the live fluoroscopic image and each corresponding image pair, and
overlay the angiographic overlay image data over the live fluoroscopic image.