Medical observation system, control device, and control method
Provided are a medical observation system, a control device, and a control method that are configured to prevent flickering of a portion to be observed of an object to be observed and reduce the size of the device. The medical observation system 1 includes an image sensor 212 , a second control unit 94 that causes a light source device 3 to simultaneously emit second visible light and excitation light, and an image processing unit 91 that generates a fluorescence image based on a first pixel value that is output from a pixel in which a first filter is arranged and that is contained in image data and a background image based on a second pixel value that is output from a pixel in which a second filter is arranged.
1. A medical observation system comprising:
a light source device configured to emit, to an object, first visible light and second visible light having different wavelength bands, and excitation light exciting a fluorescent substance and causing emission of fluorescence;
an image sensor including: a pixel portion including a plurality of pixels; a first filter configured to transmit the first visible light and the fluorescence; and a second filter configured to transmit the second visible light and the fluorescence, each of the first filter and the second filter being provided on a light receiving surface of each of the plurality of pixels, the image sensor being configured to capture at least one of reflected light of at least one of the first visible light and the second visible light reflected from the object and the fluorescence to generate image data; and
circuitry configured to
control the light source device to simultaneously emit the second visible light and the excitation light; and
generate a fluorescence image based on a first pixel value included in the image data and output from a pixel in which the first filter is arranged,
subtract, from a second pixel value output from a pixel in which the second filter is arranged, a multiplication result obtained by multiplying a value obtained by dividing a spectral sensitivity of the second filter to a fluorescence wavelength by a spectral sensitivity of the first filter to the fluorescence wavelength by the first pixel value, and
generate a background image based on a result of subtraction.
2. The medical observation system according to claim 1 , comprising a cut filter provided on an incident side of the image sensor and configured to transmit the reflected light and the fluorescence while blocking the excitation light.
3. The medical observation system according to claim 1 , wherein the circuitry is configured to
change the second pixel value based on the first pixel value, and
generate the background image based on the changed second pixel value.
4. The medical observation system according to claim 3 , wherein the circuitry is configured to
subtract the first pixel value from the second pixel value, and
generate the background image based on a result of subtraction.
5. The medical observation system according to claim 1 , wherein the circuitry is configured to
control, in a first observation mode, the light source device to emit the first visible light and the second visible light, and
control, in a second observation mode, the light source device to emit the second visible light and the excitation light.
6. The medical observation system according to claim 1 , wherein
the light source device is configured to emit third visible light having a wavelength band different from those of the first visible light and the second visible light,
the image sensor includes a third filter configured to transmit the third visible light and the fluorescence, and
the image sensor is configured to generate the background image based on a third pixel value output from a pixel in which the third filter is arranged and the second pixel value, the third pixel value and the second pixel value being included in image data.
7. The medical observation system according to claim 6 , wherein
the first visible light is light in a red wavelength band,
the second visible light is one of light in a green wavelength band and light in a blue wavelength band,
the third visible light is the other of the light in a green wavelength band and the light in a blue wavelength band,
the first filter is a red filter configured to transmit the light in a red wavelength band and the fluorescence,
the second filter is one of a green filter and a blue filter configured to transmit one of the light in a green wavelength band and the light in a blue wavelength band, and further to transmit the fluorescence, and
the third filter is the other of the green filter and the blue filter configured to transmit other of the light in a green wavelength band and the light in a blue wavelength band, and further to transmit the fluorescence.
8. The medical observation system according to claim 7 , wherein
the light source device includes:
a first light source configured to emit the light in a red wavelength band;
a second light source configured to emit the light in a green wavelength band;
a third light source configured to emit the light in a blue wavelength band; and
a fourth light source configured to emit the excitation light, and the circuitry is configured to
control the first light source, the second light source, and the third light source to emit light, in a white light observation mode for observation with white light, and
control the second light source, the third light source, and the fourth light source to emit light, in a fluorescence observation mode for observation of the fluorescence.
9. The medical observation system according to claim 1 , wherein the circuitry is configured to generate a composite image in which the background image and the fluorescence image are combined.
10. The medical observation system according to claim 1 , wherein the circuitry is configured to perform grayscale processing on at least one of the background image and the fluorescence image.
11. The medical observation system according to claim 1 , wherein the circuitry is configured to perform colorization on at least one of the background image and the fluorescence image.
12. The medical observation system according to claim 1 , wherein
the fluorescent substance is indocyanine green, and
the excitation light has a center wavelength of 740 nm.
13. The medical observation system according to claim 1 further comprising:
a support configured to turnably support a medical imaging device including the image sensor; and
a base portion configured to turnably hold a base end portion of the support, the base portion being movable on a floor surface.
14. The medical observation system according to claim 1 further comprising
an insertion section configured to be insertable into a subject and including an optical system focusing the reflected light and the fluorescence to form an object image on a light receiving surface of the image sensor.
15. The medical observation system according to claim 14 , wherein the insertion section is detachable from a medical imaging device including the image sensor.
16. A medical observation system comprising:
a light source device configured to emit, to an object, first visible light and second visible light having different wavelength bands, and excitation light exciting a fluorescent substance and causing emission of fluorescence;
a dichroic prism configured to split reflected light of at least one of the first visible light and the second visible light reflected from the object and the fluorescence into a plurality of wavelength bands;
a plurality of image sensors configured to receive light beams of the plurality of wavelength bands split by the dichroic prism and generate a plurality of pieces of image data including a first piece of image data generated from the first visible light and the fluorescence and a second piece of image data generated the second visible light and the fluorescence;
circuitry configured to
control the light source device to simultaneously emit the second visible light and the excitation light;
generate a fluorescence image based on the plurality of pieces first piece of image data;
subtract, from the second piece of image data, a multiplication result obtained by multiplying a value obtained by dividing a spectral sensitivity to a fluorescence wavelength of a second surface of the dichroic prism that directs the second visible light to a corresponding sensor by a spectral sensitivity to the fluorescence wavelength of a first surface of the dichroic prism that directs the first visible light to a corresponding sensor; and
generate a background image based on a result of subtraction.
17. A control device for controlling a light source device and a medical imaging device,
the light source device being configured to emit, to an object, first visible light and second visible light having different wavelength bands, and excitation light exciting a fluorescent substance and causing emission of fluorescence,
the medical imaging device including a pixel portion having a plurality of pixels, a first filter configured to transmit the first visible light and the fluorescence, and a second filter configured to transmit the second visible light and the fluorescence, each of the first filter and the second filter being provided on a light receiving surface of each of the plurality of pixels, the medical imaging device being configured to capture at least one of reflected light of at least one of the first visible light and the second visible light reflected from the object and the fluorescence to generate image data, the control device comprising:
circuitry configured to
control the light source device to simultaneously emit the second visible light and the excitation light;
generate a fluorescence image based on a first pixel value included in the image data and output from a pixel in which the first filter is arranged;
subtract, from a second pixel value output from a pixel in which the second filter is arranged, a multiplication result obtained by multiplying a value obtained by dividing a spectral sensitivity of the second filter to a fluorescence wavelength by a spectral sensitivity of the first filter to the fluorescence wavelength by the first pixel value; and
generate a background image based on a result of subtraction.
18. A control method executed by a control device for controlling a light source device and a medical imaging device,
the light source device being configured to emit, to an object, first visible light and second visible light having different wavelength bands, and excitation light exciting a fluorescent substance and causing emission of fluorescence,
the medical imaging device including a pixel portion having a plurality of pixels, a first filter configured to transmit the first visible light and the fluorescence, and a second filter configured to transmit the second visible light and the fluorescence, each of the first filter and the second filter being provided on a light receiving surface of each of the plurality of pixels, the medical imaging device being configured to capture at least one of reflected light of at least one of the first visible light and the second visible light reflected from the object and the fluorescence to generate image data, the control method comprising:
controlling the light source device to simultaneously emit the second visible light and the excitation light;
generating a fluorescence image based on a first pixel value included in the image data and output from a pixel in which the first filter is arranged;
subtracting, from a second pixel value output from a pixel in which the second filter is arranged, a multiplication result obtained by multiplying a value obtained by dividing a spectral sensitivity of the second filter to a fluorescence wavelength by a spectral sensitivity of the first filter to the fluorescence wavelength by the first pixel value; and
generating a background image based on a result of subtraction.
19. The control method according to claim 18 , further comprising performing colorization on at least one of the background image and the fluorescence image.
20. A non-transitory computer readable storage device having computer readable instructions that when executed by circuitry cause the circuitry to perform the control method of claim 18 .