Hyperspectral imaging in a light deficient environment
An endoscopic imaging system for use in a light deficient environment includes an imaging device having a tube, one or more image sensors, and a lens assembly including at least one optical elements that corresponds to the one or more image sensors. The endoscopic system includes a display for a user to visualize a scene and an image signal processing controller. The endoscopic system includes a light engine having an illumination source generating one or more pulses of electromagnetic radiation and a lumen transmitting one or more pulses of electromagnetic radiation to a distal tip of an endoscope.
1 . An endoscopic system for use in a light deficient environment, the system comprising:
an endoscope;
an image sensor comprising a pixel array that detects electromagnetic radiation and reads out image data; and
an emitter comprising a plurality of sources of electromagnetic radiation that emit a plurality of pulses of electromagnetic radiation, wherein the plurality of sources comprises:
a plurality of hyperspectral sources of electromagnetic radiation, and
a secondary source of electromagnetic radiation comprising one or more of:
a visible source of electromagnetic radiation,
a fluorescent excitation source of electromagnetic radiation, and
a mapping source of electromagnetic radiation; and
a controller that is in communication with the image sensor and the emitter, wherein the controller is configured to synchronize operation of the image sensor and the emitter;
wherein the controller instructs the emitter to pulse within a plurality of frame periods of the image sensor;
wherein the emitter pulses the two or more hyperspectral sources of the plurality of hyperspectral sources of electromagnetic radiation simultaneously for one or more frame periods of the plurality of frame periods;
wherein the two or more hyperspectral sources of the plurality of hyperspectral sources of electromagnetic radiation each pulse different wavelengths of electromagnetic radiation.
2 . The endoscopic system of claim 1 , wherein the pixel array detects the electromagnetic radiation and reads out image data in response to each of a plurality of pulse durations, and wherein the light engine emits one or more pulses of electromagnetic radiation during each pulse duration of the plurality of pulse durations.
3 . The endoscopic system of claim 1 , wherein the pixel array reads out the image data for generating a plurality of exposure frames, and wherein each of the plurality of exposure frames corresponds to one pulse duration of the plurality of pulse durations.
4 . The endoscopic system of claim 3 , wherein data from two or more of the plurality of exposure frames are displayed to a user as an overlay image, and wherein the overlay image comprises a depiction of the one or more of a dimension, surface area, or relative position of one or more of a surgical tool, anatomical structure, or object within a scene.
5 . The endoscopic system of claim 3 , wherein a portion of the plurality of exposure frames comprises a hyperspectral exposure frame comprising hyperspectral imaging data generated by the plurality of hyperspectral sources of electromagnetic radiation, and wherein the hyperspectral imaging data comprises information from a single spectrum or a combination of spectrums of pulsed electromagnetic radiation.
6 . The endoscopic system of claim 1 , wherein the plurality of hyperspectral sources of electromagnetic radiation are tuned to emit electromagnetic radiation within a waveband, the plurality of sources comprising two or more of:
a source tuned to emit only electromagnetic radiation within a waveband from about 425 nm to about 475 nm;
a source tuned to emit only electromagnetic radiation within a waveband from about 520 nm to about 545 nm;
a source tuned to emit only electromagnetic radiation within a waveband from about 625 nm to about 645 nm;
a source tuned to emit only electromagnetic radiation within a waveband from about 760 nm to about 795 nm;
a source tuned to emit only electromagnetic radiation within a waveband from about 795 nm to about 815 nm;
a source tuned to emit only electromagnetic radiation within a waveband from about 370 nm to about 420 nm; or
a source tuned to emit only electromagnetic radiation within a waveband from about 600 nm to about 670 nm.
7 . The endoscopic system of claim 1 , wherein the plurality of hyperspectral sources of electromagnetic radiation comprises a plurality of spectral sources of electromagnetic radiation, the spectral sources comprising two or more of:
a first spectral source tuned to emit the first spectral waveband, wherein the first spectral waveband comprises electromagnetic radiation from about 513 nm to about 545 nm;
a second spectral source tuned to emit the second spectral waveband, wherein the second spectral waveband comprises electromagnetic radiation from about 565 nm to about 585 nm;
a third spectral source tuned to emit a third spectral waveband, wherein the third spectral waveband comprises electromagnetic radiation from about 900 nm to about 1000 nm;
a fourth spectral source tuned to emit a fourth spectral waveband, wherein the fourth spectral waveband comprises electromagnetic radiation from about 770 nm to about 795 nm; and
a fifth spectral source tuned to emit a fifth spectral waveband, wherein the fifth spectral waveband comprises electromagnetic radiation from about 790 nm to about 815 nm.
8 . The endoscopic system of claim 1 , wherein the secondary source of electromagnetic radiation is the mapping source, and wherein the mapping source is configured to emit electromagnetic radiation for generating mapping data, wherein the mapping data comprises one or more of:
surface area information about an object,
a size of one or more surgical tools,
a size of one or more tissue structures,
positional information about one or more objects,
a distance between two or more objects.
9 . The endoscopic system of claim 1 , wherein the controller is further configured to:
provide imaging data acquired from the image sensor to an algorithm to identify a location of the target tissue within a scene; and
render an overlay frame highlighting the target tissue based on an output received from the algorithm, wherein the highlight of the target tissue is encoded to any color selected by either an overlay algorithm or a user.
10 . The endoscopic system of claim 1 , further comprising one or more filters, the one or more filters comprising:
a filter that prevents electromagnetic radiation between 770 nm and 795 nm from irradiating the pixel array of the image sensor;
a filter that prevents electromagnetic radiation between 790 nm and 815 nm from irradiating the pixel array of the image sensor;
a filter that prevents electromagnetic radiation between 760 nm and 785 nm from irradiating the pixel array of the image sensor; or
a filter that prevents electromagnetic radiation between 800 nm and 850 nm from irradiating the pixel array of the image sensor.
11 . The endoscopic system of claim 1 , wherein the image sensor comprises a first image sensor and a second image sensor, and wherein dimensional information is calculated based on data output by the first image sensor and the second image sensor to produce a three-dimensional image.
12 . The endoscopic system of claim 1 , wherein the controller instructs the emitter to operate during a blanking period of the image sensor, and wherein the controller instructs the emitter to only pulse during the blanking period; and
wherein the image sensor comprises a plurality of blanking periods, and wherein each blanking period of the plurality of blanking periods is variable in duration.
13 . The endoscopic system of claim 1 , wherein the image sensor comprises a readout period during which the emitter does not pulse, and wherein the system is configured to adjust the duration of the readout period.
14 . The endoscopic system of claim 1 , wherein the controller is configured to modulate a duration and/or a magnitude of one or more of the plurality of pulses of electromagnetic radiation while maintaining a consistent display frame rate of images read out to a display.
15 . The endoscopic system of claim 1 , wherein the controller is further configured to:
determine one or more specified wavebands of electromagnetic radiation selected by a user,
instruct the emitter to pulse a plurality of pulses of electromagnetic radiation according to the one or more specified wavebands,
generate a hyperspectral exposure frame,
provide the hyperspectral exposure frame to a corresponding system to classify a tissue or material captured in the hyperspectral exposure frame,
overlay the hyperspectral exposure frame on a black and white image frame or an RGB image frame to create an overlay frame, and
display the overlay frame on a display.
16 . The endoscopic system of claim 1 , wherein the plurality of hyperspectral sources of electromagnetic radiation is configured to pulse a plurality of wavebands of electromagnetic radiation, wherein each of the plurality of wavebands comprises a width of 10 nm or less.
17 . The endoscopic system of claim 1 , wherein the plurality of hyperspectral sources of electromagnetic radiation pulses electromagnetic radiation that penetrates tissue such that the image sensor may receive reflected electromagnetic radiation to capture images at a depth of 5 nm, 8 nm, or 10 nm.
18 . The endoscopic system of claim 1 , wherein the image sensor reads out image data obtained in response to the emitter pulsing the two or more hyperspectral sources of electromagnetic radiation simultaneously.
19 . The endoscopic system of claim 18 , wherein the system analyzes the image data obtained in response to the emitter pulsing the two or more hyperspectral sources of electromagnetic radiation simultaneously to determine a classification of a tissue or a tissue state of the tissue.
20 . The endoscopic system of claim 1 , wherein a duration of each of the plurality of frame periods is variable.