System and method, in particular for microscopes and endoscopes, for creating HDR monochrome images of a fluorescing fluorophore
A method and system for computing an HDR image ( 38 ) from a digital color input image ( 4 ) of an object ( 30 ) containing a fluorescing fluorophore ( 22 ) acquires the input image using a color camera ( 2 ) having at least two different types ( 16, 17, 18 ) of color sensor ( 8 ), such as an R, G and B sensor. The input image may be recorded in a co-sensing wavelength band ( 64, 66, 68 ) wherein different spectral responsivities ( 58, 60, 62 ) of the different types of color sensor overlap. The input image comprises different digital monochrome input images ( 6 ), each recorded by a different type of color sensor. Light incident on the camera may be filtered using a band-pass filter ( 32 ) having a tunable pass band ( 34 ) which defines the co-sensing wavelength band and may be adjusted depending on spectral responsivities of the color sensors, the fluorophore, and characteristics of the monochrome input images.
1. An image acquisition system ( 1 ) for a microscope or an endoscope, for processing digital input images ( 4 , 6 ) of a fluorophore ( 22 ) having a fluorescence spectrum ( 82 , 84 ), the image acquisition system ( 1 ) comprising:
a color camera ( 2 ),
the color camera ( 2 ) having at least two different types ( 16 , 17 , 18 ) of color sensor ( 8 ) at a plurality of input pixels ( 10 ), each type ( 16 , 17 , 18 ) of color sensor ( 8 ) generating a different digital monochrome input image ( 6 ) comprising the input pixels ( 10 ) and having a different spectral responsivity ( 58 , 60 , 62 ),
the color camera ( 2 ) further comprising at least one co-sensing wavelength band ( 64 , 66 , 68 ) in which the spectral responsivities of at least two types of color sensor overlap, wherein the co-sensing wavelength band ( 64 , 66 , 68 ) overlaps with the fluorescence spectrum ( 82 , 84 ) of the fluorophore ( 22 );
an HDR image generator ( 40 ) coupled to the color camera ( 2 );
wherein the HDR image generator ( 40 ) is configured to compute a digital monochrome HDR output image ( 38 ) from the at least two digital monochrome input images ( 6 );
an optical band-pass filter ( 32 ) located in front of the color camera ( 2 ), the optical band-pass filter ( 32 ) having a pass band ( 34 ), the pass band ( 34 ) defining the co-sensing wavelength band ( 64 , 66 , 68 ); and
a memory section ( 80 ) comprising a representation of the spectral responsivities ( 58 , 60 , 62 ) of the different types ( 14 , 16 , 18 ) of color sensor ( 8 ) of the color camera ( 2 ), and wherein the image acquisition system ( 1 ) is configured to automatically adjust the pass band ( 34 ) depending on the spectral responsivities ( 58 , 60 , 62 ).
2. The image acquisition system ( 1 ) according to claim 1 , wherein the pass band ( 34 ) of the band-pass filter ( 32 ) overlaps with the fluorescence spectrum ( 82 , 84 ) of the fluorophore ( 22 ).
3. The image acquisition system ( 1 ) according to claim 1 , the band-pass filter ( 32 ) comprising a tunable pass band ( 34 ), the pass band ( 34 ) being tunable with respect to at least one filter characteristic ( 77 ), the at least one filter characteristic ( 77 ) comprising at least one of: a center frequency, at least one cut-off frequency, a roll-off ( 76 ), a bandwidth ( 75 ), and a Q-factor.
4. An image acquisition system ( 1 ) for a microscope or an endoscope, for processing digital input images ( 4 , 6 ) of a fluorophore ( 22 ) having a fluorescence spectrum ( 82 , 84 ), the image acquisition system ( 1 ) comprising:
a color camera ( 2 ),
the color camera ( 2 ) having at least two different types ( 16 , 17 , 18 ) of color sensor ( 8 ) at a plurality of input pixels ( 10 ), each type ( 16 , 17 , 18 ) of color sensor ( 8 ) generating a different digital monochrome input image ( 6 ) comprising the input pixels ( 10 ) and having a different spectral responsivity ( 58 , 60 , 62 ),
the color camera ( 2 ) further comprising at least one co-sensing wavelength band ( 64 , 66 , 68 ) in which the spectral responsivities of at least two types of color sensor overlap, wherein the co-sensing wavelength band ( 64 , 66 , 68 ) overlaps with the fluorescence spectrum ( 82 , 84 ) of the fluorophore ( 22 );
an HDR image generator ( 40 ) coupled to the color camera ( 2 );
wherein the HDR image generator ( 40 ) is configured to compute a digital monochrome HDR output image ( 38 ) from the at least two digital monochrome input images ( 6 );
an optical band-pass filter ( 32 ) located in front of the color camera ( 2 ), the optical band-pass filter ( 32 ) having a pass band ( 34 ), the pass band ( 34 ) defining the co-sensing wavelength band ( 64 , 66 , 68 );
the band-pass filter ( 32 ) comprising a tunable pass band ( 34 ), the pass band ( 34 ) being tunable with respect to at least one filter characteristic ( 77 ), the at least one filter characteristic ( 77 ) comprising at least one of: a center frequency, at least one cut-off frequency, a roll-off ( 76 ), a bandwidth ( 75 ), and a Q-factor; and
a filter controller ( 78 ) coupled to the HDR image generator ( 40 ) for modifying the pass band ( 34 ) of the band-pass filter ( 32 ) depending on image characteristics of the digital monochrome input images ( 6 ) and/or the digital monochrome HDR output image ( 38 ).
5. An image acquisition system ( 1 ) for a microscope or an endoscope, for processing digital input images ( 4 , 6 ) of a fluorophore ( 22 ) having a fluorescence spectrum ( 82 , 84 ), the image acquisition system ( 1 ) comprising:
a color camera ( 2 ),
the color camera ( 2 ) having at least two different types ( 16 , 17 , 18 ) of color sensor ( 8 ) at a plurality of input pixels ( 10 ), each type ( 16 , 17 , 18 ) of color sensor ( 8 ) generating a different digital monochrome input image ( 6 ) comprising the input pixels ( 10 ) and having a different spectral responsivity ( 58 , 60 , 62 ),
the color camera ( 2 ) further comprising at least one co-sensing wavelength band ( 64 , 66 , 68 ) in which the spectral responsivities of at least two types of color sensor overlap, wherein the co-sensing wavelength band ( 64 , 66 , 68 ) overlaps with the fluorescence spectrum ( 82 , 84 ) of the fluorophore ( 22 );
an HDR image generator ( 40 ) coupled to the color camera ( 2 );
wherein the HDR image generator ( 40 ) is configured to compute a digital monochrome HDR output image ( 38 ) from the at least two digital monochrome input images ( 6 );
an optical band-pass filter ( 32 ) located in front of the color camera ( 2 ), the optical band-pass filter ( 32 ) having a pass band ( 34 ), the pass band ( 34 ) defining the co-sensing wavelength band ( 64 , 66 , 68 );
the band-pass filter ( 32 ) comprising a tunable pass band ( 34 ), the pass band ( 34 ) being tunable with respect to at least one filter characteristic ( 77 ), the at least one filter characteristic ( 77 ) comprising at least one of: a center frequency, at least one cut-off frequency, a roll-off ( 76 ), a bandwidth ( 75 ), and a Q-factor; and
a control loop ( 108 ) for automatically modifying the pass band ( 34 ) depending on an image characteristic of at least one of the digital monochrome input images ( 6 ) and/or the digital monochrome HDR output image ( 38 ).
6. A medical observation apparatus ( 3 ) comprising:
an image acquisition system ( 1 ) for processing digital input images ( 4 , 6 ) of a fluorophore ( 22 ) having a fluorescence spectrum ( 82 , 84 ), the image acquisition system ( 1 ) comprising:
a color camera ( 2 ),
the color camera ( 2 ) having at least two different types ( 16 , 17 , 18 ) of color sensor ( 8 ) at a plurality of input pixels ( 10 ), each type ( 16 , 17 , 18 ) of color sensor ( 8 ) generating a different digital monochrome input image ( 6 ) comprising the input pixels ( 10 ) and having a different spectral responsivity ( 58 , 60 , 62 ),
the color camera ( 2 ) further comprising at least one co-sensing wavelength band ( 64 , 66 , 68 ) in which the spectral responsivities of at least two types of color sensor overlap, wherein the co-sensing wavelength band ( 64 , 66 , 68 ) overlaps with the fluorescence spectrum ( 82 , 84 ) of the fluorophore ( 22 );
an HDR image generator ( 40 ) coupled to the color camera ( 2 );
wherein the HDR image generator ( 40 ) is configured to compute a digital monochrome HDR output image ( 38 ) from the at least two digital monochrome input images ( 6 );
an optical band-pass filter ( 32 ) located in front of the color camera ( 2 ), the optical band-pass filter ( 32 ) having a pass band ( 34 ), the pass band ( 34 ) defining the co-sensing wavelength band ( 64 , 66 , 68 ); and
a memory section ( 80 ) comprising a representation of the spectral responsivities ( 58 , 60 , 62 ) of the different types ( 14 , 16 , 18 ) of color sensor ( 8 ) of the color camera ( 2 ), and wherein the image acquisition system ( 1 ) is configured to automatically adjust the pass band ( 34 ) depending on the spectral responsivities ( 58 , 60 , 62 );
wherein the medical observation apparatus ( 3 ) is a microscope or an endoscope.
7. A method for creating a digital monochrome HDR output image ( 38 ) of a fluorescing fluorophore ( 22 ) from at least two digital monochrome input images ( 6 ) each having a plurality of input pixels ( 10 ), the method comprising the steps of:
acquiring the at least two digital monochrome input images ( 6 ) of the fluorescing fluorophore ( 22 ) using at least two different types ( 16 , 17 , 18 ) of color sensor ( 8 ), the at least two different types of color sensor having a different spectral responsivity ( 58 , 60 , 62 ) and comprising at least one co-sensing wavelength band ( 64 , 66 , 68 ) in which the spectral responsivities of at least two types of color sensor overlap;
selecting the fluorescing fluorophore ( 22 ) to have a fluorescence spectrum ( 82 , 84 ) which overlaps with the co-sensing wavelength band; and
computing a digital monochrome HDR output image ( 38 ) from the at least two digital monochrome input images ( 6 ).
8. The method according to claim 7 , further comprising the step of filtering the fluorescent light from the fluorophore ( 22 ) directed to the color sensors ( 8 ) using an optical band-pass filter ( 32 ) having a pass band ( 34 ) which overlaps with the co-sensing wavelength band ( 64 , 66 , 68 ).
9. The method according to claim 7 , further comprising the step of modifying the pass band ( 34 ) to increase contrast in the digital monochrome HDR output image ( 38 ).
10. The method according to claim 9 , further comprising the step of automatically modifying the pass band ( 34 ) depending on an image characteristic of at least one of the digital monochrome input images ( 6 ) and/or the digital monochrome HDR output image ( 38 ).
11. The method according to claim 9 , further comprising the step of modifying the pass band ( 34 ) depending on a difference between the spectral responsivities ( 58 , 60 , 62 ) of the at least two different types ( 16 , 17 , 18 ) of color sensor ( 8 ) within the pass band ( 34 ).
12. The method according to claim 9 , further comprising the step of recording the at least two digital monochrome input images ( 6 ) using the at least two different types ( 16 , 17 , 18 ) of color sensor ( 8 ).
13. A non-transitory computer-readable medium storing a program causing a computer to execute the method according to claim 7 .