Wide dynamic range using a monochrome image sensor for fluorescence imaging

1. A system comprising:

an emitter that cycles a plurality of independent electromagnetic sources on and off to emit a plurality of pulses of electromagnetic radiation according to a variable pulse cycle, wherein the plurality of independent electromagnetic sources comprises:

a visible source that emits a visible wavelength of electromagnetic radiation;

a first fluorescence source that emits electromagnetic radiation within a waveband from about 770 nm to about 795 nm; and

a second fluorescence source that emits electromagnetic radiation within a waveband from about 790 nm to about 815 nm;

an image sensor comprising a pixel array for sensing reflected electromagnetic radiation, wherein the pixel array comprises a plurality of pixels each configurable as a short exposure pixel or a long exposure pixel; and

a controller comprising a processor in electrical communication with the image sensor and the emitter that synchronizes operations of the image sensor and the emitter to generate a video output;

wherein the controller instructs the emitter to separately actuate the visible source, the first fluorescence source, and the second fluorescence source according to the variable pulse cycle; and

wherein the controller instructs the emitter to change a sequence of the variable pulse cycle based on user input and a desired video output.

2. The system of claim 1 , wherein the pixel array comprises a plurality of short exposure pixels and a plurality of long exposure pixels arranged in a checkerboard pattern such that a short exposure pixel is located adjacent to a long exposure pixel.

3. The system of claim 1 , wherein:

each of the plurality of pixels of the pixel array comprises a transfer gate transistor;

each transfer gate transistor is in electrical communication with a TX signal; and

the TX signal provides a global operation for the transfer gate transistors of the plurality of pixels of the pixel array.

4. The system of claim 3 , wherein:

the TX signal comprises a TX1 signal and a TX2 signal;

at least two pixels of the plurality of pixels of the pixel array share a floating diffusion in a horizontal direction in a two-way pixel share;

the TX1 signal communicates with transfer gate transistors of pixels located on a first side of the two-way pixel share on odd rows and with transfer gate transistors of pixels located on a second side of the two-way pixel share on even rows; and

the TX2 signal communicates with transfer gate transistors of pixels located on the second side of the two-way pixel share on odd rows and with transfer gate transistors of pixels located on the first side of the two-way pixel share on even rows.

5. The system of claim 1 , wherein the image sensor performs horizontal binning during a charge period of the pixel array.

6. The system of claim 1 , wherein a pixel grouping of the pixel array shares a floating diffusion, wherein the pixel grouping comprises two pixels or four pixels.

7. The system of claim 1 , wherein the pixel array is disposed on a first substrate of the image sensor and supporting circuitry for the pixel array is disposed remotely on a second substrate of the image sensor.

8. The system of claim 1 , wherein the image sensor is configured to generate a plurality of exposure frames, wherein each of the plurality of exposure frames corresponds to a pulse of electromagnetic radiation emitted by the emitter.

9. The system of claim 1 , wherein the pixel array accumulates electromagnetic radiation emitted by the emitter during a blanking period of the image sensor, and wherein the pixel array reads out data during a readout period of the image sensor.

10. The system of claim 1 , wherein at least one of the first fluorescence source and the second fluorescence source emits electromagnetic radiation in an excitation wavelength that causes one or more reagents to fluoresce at a wavelength that is different from the excitation wavelength.

11. The system of claim 1 , wherein data sensed by the image sensor is used to generate a plurality of exposure frames, wherein the plurality of exposure frames comprises a color exposure frame sensed in response to an emission by the visible source, a first fluorescence expose frame sensed in response to an emission by the first fluorescence source, and a second fluorescence exposure frame sensed in response to an emission by the second fluorescence source.

12. The system of claim 1 , wherein the variable pulse cycle comprises a plurality of independent pulse durations each assigned to one or more of the visible source, the first fluorescence source, or the second fluorescence source, and wherein the emitter is configured to emit, during one of the plurality of independent pulse durations, a plurality of sub-pulses of electromagnetic radiation.

13. The system of claim 1 , wherein the variable pulse cycle comprises a plurality of independent pulse emissions, and wherein at least one of the plurality of independent pulse emissions comprises electromagnetic radiation emitted at two or more wavelengths simultaneously.

14. The system of claim 1 , wherein the variable pulse cycle comprises a plurality of independent pulse emissions, and wherein the pixel array senses reflected electromagnetic radiation to generate an exposure frame in response to each of the plurality of independent pulse emissions, and wherein the system further comprises a display for displaying two or more exposure frames as an image frame.

15. The system of claim 1 , wherein the pixel array senses the reflected electromagnetic radiation to generate an exposure frame in response to each pulse of electromagnetic radiation pulsed in the variable pulse cycle, and wherein one or more of:

the pixel array senses a first fluorescence frame in response to an emission by the first fluorescence source, wherein the electromagnetic radiation pulsed by the first fluorescence source is an excitation wavelength for fluorescing a reagent, and wherein the first fluorescence frame comprises an indication of a location of the reagent within a scene; or

the pixel array senses a second fluorescence frame in response to an emission by the second fluorescence source, wherein the electromagnetic radiation pulsed by the second fluorescence source is an excitation wavelength for fluorescing the reagent, and wherein the second fluorescence frame comprises an indication of a location of the reagent within the scene.

16. The system of claim 15 , wherein the controller is configured to provide one or more of the first fluorescence frame or the second fluorescence frame to a corresponding system that determines a location of a tissue structure based on one or more of the first fluorescence frame or the second fluorescence frame.

17. The system of claim 16 , wherein the controller is configured to:

receive the location of the tissue structure from the corresponding system;

generate an overlay frame comprising the location of the tissue structure; and

combine the overlay frame with a color image frame depicting the scene to indicate the location of the tissue structure within the scene.

18. The system of claim 1 , wherein the controller is configured to synchronize timing of the pulses of electromagnetic radiation of the variable pulse cycle to be emitted during a blanking period of the image sensor, wherein the blanking period corresponds to a time between a readout of a last row of active pixels in the pixel array and a beginning of a next subsequent readout of active pixels in the pixel array.

19. The system of claim 1 , wherein the controller adjusts one or more of the sequence of the variable pulse cycle, a duration of one or more pulses in the variable pulse cycle, or an intensity of one or more pulses in the variable pulse cycle based on a threshold, wherein the threshold determines proper illumination of a scene in a light deficient environment.

20. The system of claim 1 , wherein two or more pulses of electromagnetic radiation emitted by the emitter result in two or more instances of reflected electromagnetic radiation that are sensed by the pixel array to generate two or more exposure frames that are combined to form an image frame, and wherein a sequence of exposure frames is adjusted when the controller adjusts the sequence of the variable pulse cycle.

21. The system of claim 1 , wherein one or more of the first fluorescence source or the second fluorescence source emits an excitation wavelength for fluorescing a reagent, and wherein at least a portion of the reflected electromagnetic radiation sensed by the image sensor is a relaxation wavelength of the reagent.

22. The system of claim 21 , wherein the image sensor is configured to sense the relaxation wavelength of the reagent to generate a fluorescence exposure frame, and wherein the controller is configured to provide the fluorescence exposure frame to a corresponding system that identifies one or more structures in a body based on the fluorescence exposure frame.

23. The system of claim 22 , wherein the one or more structures in the body comprise one or more of a nerve, a ureter, a blood vessel, an artery, a blood flow, cancerous tissue, or a tumor.

24. The system of claim 1 , further comprising a first filter that filters electromagnetic radiation within a waveband from about 770 nm to about 795 nm.

25. The system of claim 1 , further comprising a second filter that filters electromagnetic radiation within a waveband from about 790 nm to about 815 nm.

26. The system of claim 1 , further comprising a first filter that filters electromagnetic radiation within a waveband from about 770 nm to about 795 nm and a second filter that filters electromagnetic radiation within a waveband from about 790 nm to about 815 nm.

27. The system of claim 1 , further comprising one or more filters that allow electromagnetic radiation having a wavelength from about 790 nm to about 800 nm and above 815 nm to pass through the one or more filters to the image sensor.

28. The system of claim 1 , further comprising a polarization filter located in a path of the pulses of electromagnetic radiation emitted by the emitter.

29. The system of claim 1 , wherein the image sensor comprises a first image sensor and a second image sensor such that the image sensor can generate a three-dimensional image.

30. The system of claim 1 , wherein the emitter is configured to emit the variable pulse cycle repeatedly sufficient for generating the video stream, and wherein each image frame in the video stream comprises data from a plurality of exposure frames, and wherein each of the plurality exposure frames corresponds to a pulse of electromagnetic radiation emitted in the variable pulse cycle.