Pulsed illumination in a hyperspectral, fluorescence and laser mapping imaging system
Pulsed hyperspectral, fluorescence, and laser mapping imaging in a light deficient environment is disclosed. A system includes an emitter for emitting pulses of electromagnetic radiation and an image sensor comprising a pixel array for sensing reflected electromagnetic radiation. The system includes a controller configured to synchronize timing of the emitter and the image sensor. The system is such that at least a portion of the pulses of electromagnetic radiation emitted by the emitter comprises one or more of a hyperspectral emission, a fluorescence emission, or a laser mapping pattern.
1 . An endoscopic system for providing visualization to a light deficient environment comprising:
an emitter for emitting a plurality of emissions of electromagnetic radiation, wherein the plurality of emissions are a plurality of pulses of electromagnetic radiation;
an image sensor comprising a pixel array, wherein data is read out from the pixel array corresponding to the plurality of pulses of electromagnetic radiation; and
a processor for performing image signal processing, wherein the processor receives the data from the image sensor and generates a plurality of data frames;
a controller in electronic communication with the emitter and the image sensor;
wherein the plurality of emissions of electromagnetic radiation comprises a visible emission of electromagnetic radiation and a spectral emission of electromagnetic radiation for eliciting a spectral response from a tissue, and one or more emission sources comprising:
a mapping emission comprising electromagnetic radiation for generating laser mapping data; or
a fluorescence emission comprising a fluorescence excitation wavelength of electromagnetic radiation;
wherein the controller instructs the emitter to adjust a magnitude, a duration, or a magnitude and a duration of the plurality of emissions of electromagnetic radiation emitted by the emitter;
wherein the plurality of pulses of electromagnetic radiation switch between the visible emission of electromagnetic radiation, the spectral emission, and the one or more emission sources in a pulse cycle pattern; and
wherein the plurality of data frames comprises one or more visible image frames generated from the visible emission of electromagnetic radiation, and one or more exposure frames corresponding to and generated from the spectral emission of electromagnetic radiation and the one or more emission sources.
2 . The system of claim 1 , wherein the emitter pulses the plurality of pulses of electromagnetic radiation for a duration that is during portions of two or more operational cycles of the image sensor.
3 . The system of claim 1 , wherein the magnitude, the duration, or the magnitude and the duration of the plurality of emissions of electromagnetic radiation is adjusted concurrently.
4 . The system of claim 1 , wherein the emitter pulses the mapping emission and the image sensor senses a laser mapping frame comprising the laser mapping data in response to the pulse of the mapping emission, and wherein the laser mapping data comprises data for calculating one or more of a three-dimensional topography of a scene, a dimension of one or more objects within a scene, or a distance.
5 . The system of claim 1 , wherein the spectral emission of electromagnetic radiation comprises one or more of:
electromagnetic radiation comprising a wavelength within a range from about 513 nm to about 545 nm;
electromagnetic radiation comprising a wavelength within a range from about 565 nm to about 585 nm; or
electromagnetic radiation comprising a wavelength within a range from about 900 nm to about 1000 nm.
6 . The system of claim 1 , wherein the fluorescence excitation wavelength of electromagnetic radiation comprises one or more of:
electromagnetic radiation comprising a wavelength within a range from about 770 nm to about 795 nm; or
electromagnetic radiation comprising a wavelength within a range from about 790 nm to about 815 nm.
7 . The system of claim 1 , wherein the controller is configured to synchronize timing of the plurality of emissions of electromagnetic radiation during a blanking period of the image sensor, wherein the blanking period corresponds to a time between a readout of a last row of active pixels in the pixel array and a beginning of a next subsequent readout of active pixels in the pixel array.
8 . The system of claim 7 , wherein the controller is configured to adjust a length of the blanking period to facilitate different blanking periods.
9 . The system of claim 7 , wherein controller is configured to adjust a length of a readout period to facilitate different readout periods.
10 . The system of claim 7 , wherein the emitter begins one or more pulses of the plurality of pulses of electromagnetic radiation during a readout period of the image sensor and ends the one or more pulses of the plurality of pulses of electromagnetic radiation during a readout period of a next succeeding cycle of the image sensor.
11 . The system of claim 8 , wherein the image sensor is configured to facilitate a repeating pattern of different-length blanking periods across operational cycles of the image sensor.
12 . The system of claim 1 , wherein the mapping emission comprises one or more of a raster grid of discrete points, an occupancy grid map, a dot array, vertical hashing, or horizontal hashing.
13 . The system of claim 1 , wherein the image sensor senses a plurality of frame-types in response to the emitter emitting the plurality of emissions of electromagnetic radiation, wherein the plurality of frame-types comprises:
a laser mapping frame sensed in response to the emitter emitting the mapping emission;
a spectral frame sensed in response to the emitter emitting the spectral emission; and
a fluorescence frame sensed in response to the emitter emitting the fluorescence emission.
14 . The system of claim 1 , wherein the processor is configured to execute instructions stored in non-transitory computer readable storage medium, the instructions comprising:
receiving a laser mapping frame sensed by the image sensor, wherein the laser mapping frame is sensed in response to the emitter emitting the mapping emission;
providing the laser mapping frame to a corresponding laser mapping system; and
receiving laser mapping data from the corresponding laser mapping system, wherein the laser mapping data comprises one or more of:
a three-dimensional topographical map of a scene, a dimension of one or more objects within the scene, or a distance.
15 . The system of claim 14 , wherein the instructions further comprise:
receiving a spectral frame sensed by the image sensor, wherein the spectral frame is sensed in response to the emitter emitting the spectral emission;
providing the spectral frame to a corresponding spectral system; and
receiving spectral data from the corresponding spectral system, wherein the spectral data comprises one or more of:
a predicted identity of a tissue structure within the scene; or
a location of the tissue structure within the scene.
16 . The system of claim 14 , wherein the instructions further comprise:
receiving a fluorescence frame sensed by the image sensor, wherein the fluorescence frame is sensed in response to the emitter emitting the fluorescence emission;
providing the fluorescence frame to a corresponding fluorescence system; and
receiving fluorescence data from the corresponding fluorescence system, wherein the fluorescence data comprises one or more of:
a predicted identity of a tissue structure within the scene; or
a location of the tissue structure within the scene.
17 . The system of claim 14 , wherein the instructions further comprise:
receiving the one or more visible image frames of the plurality of visible image frames;
overlaying at least a portion of the laser mapping data, including one of the exposure frames, with at least one of the visible image frames; and
overlaying at least a portion of the fluorescence data, including one of the exposure frames, with the visible image frame.
18 . The system of claim 1 , further comprising:
an endoscope comprising a handpiece and a lumen, wherein the image sensor is disposed substantially near a distal end of the lumen of the endoscope; and
a waveguide for carrying the plurality of emissions of electromagnetic radiation from the emitter to the distal end of the lumen of the endoscope;
wherein the emitter is remote from the endoscope.
19 . The system of claim 1 , wherein the plurality of emissions of electromagnetic radiation further comprises:
a luminance emission comprising a wavelength of electromagnetic radiation that is optimized for sensing luminance data with the image sensor;
a red chrominance emission comprising a wavelength of electromagnetic radiation that is optimized for sensing red-chrominance data with the image sensor; and
a blue chrominance emission comprising a wavelength of electromagnetic radiation that is optimized for sensing blue-chrominance data with the image sensor.
20 . The system of claim 1 , wherein the image signal processing further comprises:
receiving a plurality of independent frames sensed by the image sensor;
performing color correction on the plurality of independent frames;
performing edge enhancement on the plurality of independent frames; and
converting the luminance-chrominance frame to an RGB color image frame.