Image synchronization without input clock and data transmission clock in a pulsed hyperspectral, fluorescence, and laser mapping imaging system
Pulsed hyperspectral, fluorescence, and laser mapping imaging without input clock or data transmission clock is disclosed. A system includes an emitter for emitting pulses of electromagnetic radiation and an image sensor comprising a pixel array for sensing reflected electromagnetic radiation. The system includes a plurality of bidirectional data pads and a controller in communication with the image sensor. The system is such that at least a portion of the pulses of electromagnetic radiation emitted by the emitter comprises one or more of: electromagnetic radiation having a wavelength from about 513 nm to about 545 nm, from about 565 nm to about 585 nm, from about 900 nm to about 1000 nm, an excitation wavelength of electromagnetic radiation that causes a reagent to fluoresce, or a laser mapping pattern.
1. A system comprising:
an emitter comprising a plurality of sources of electromagnetic radiation each configured for emitting a plurality of pulses of electromagnetic radiation;
an image sensor comprising a pixel array for sensing reflected electromagnetic radiation;
a plurality of bidirectional data pads on the image sensor for outputting image data and receiving configuration data; and
a controller in electronic communication with the image sensor and the emitter;
wherein the plurality of sources of electromagnetic radiation comprises a spectral source of electromagnetic radiation for emitting visible and non-visible electromagnetic radiation for eliciting a spectral response from a tissue, and further comprises one or more of:
a mapping source of electromagnetic radiation for generating mapping data; or
an excitation source of electromagnetic radiation;
wherein the controller synchronizes operations of the image sensor and the emitter such that the image sensor outputs data corresponding with the plurality of pulses of electromagnetic radiation emitted by the plurality of sources of electromagnetic radiation of the emitter;
wherein each of the plurality of sources of electromagnetic radiation are independently actuatable.
2. The system of claim 1 , further comprising:
an oscillator disposed in the controller; and
a frequency detector in electronic communication with the controller to control a clock frequency of the image sensor in response to signals from the controller that correspond to the frequency of the oscillator.
3. The system of claim 1 , wherein electronic communication through each of the plurality of bidirectional pads is performed in defined phases comprising:
a rolling readout phase during which image data is output from the image sensor through the plurality of bidirectional pads to the controller;
a service line phase during which non-image data is output from the image sensor through the plurality of bidirectional pads to the controller; and
a configuration phase during which image sensor configuration data is received by the image sensor from the controller through the plurality of bidirectional pads;
wherein image sensor data output lines are reversed during the configuration phase; and
wherein a clock data recovery circuit is unlocked from an oscillator disposed in the controller when the image sensor data output lines are reversed.
4. The system of claim 3 , wherein clock signal data is transmitted during the service line phase and the configuration phase and is not transmitted during the rolling readout phase.
5. The system of claim 1 , wherein the image sensor does not comprise a dedicated input synchronization clock pad such that a total number of data pads in the image sensor is reduced.
6. The system of claim 1 , further comprising an oscillator disposed in the controller for providing a reference clock frequency to synchronize incoming data received from the image sensor.
7. The system of claim 1 , further comprising at least one transition within pixel serial data created by the pixel array of the image sensor, wherein the pixel serial data is replaced in the pixel array according to synchronization determined by clock signal data.
8. The system of claim 1 , further comprising:
a phase lock loop (PLL) comprising PLL image sensor blocks and PLL controller blocks;
data lines electrically connecting the PLL image sensor blocks and the PLL controller blocks; and
configuration lines electrically connecting the PLL image sensor blocks and the PLL controller blocks;
wherein an oscillator disposed in the controller and a data recovery circuit are used to lock on to incoming data received by the controller from the image sensor.
9. The system of claim 1 , wherein the image sensor is configured to generate a plurality of exposure frames, wherein each of the plurality of exposure frames corresponds to a pulse of electromagnetic radiation emitted by at least one of the plurality of sources of electromagnetic radiation of the emitter.
10. The system of claim 9 , wherein the pixel array of the image sensor senses reflected electromagnetic radiation to generate the plurality of exposure frames during a readout period of the pixel array, wherein the readout period is a duration of time when active pixels in the pixel array are read.
11. The system of claim 1 , further comprising:
a voltage controlled oscillator;
a phase lock loop based on a charge pump for driving the voltage controlled oscillator; and
a phase lock loop based on a digital-to-analog converter for driving the voltage controlled oscillator.
12. The system of claim 1 , wherein at least a portion of the pulses of electromagnetic radiation emitted by the emitter is a fluorescence emission emitted by the excitation source for exciting a fluorescent reagent, wherein the fluorescence emission results in a fluorescence exposure frame created by the image sensor, and wherein the controller is configured to provide the fluorescence exposure frame to a corresponding system that determines a location of a tissue structure within a scene based on the fluorescence exposure frame.
13. The system of claim 12 , wherein the controller is further configured to:
receive the location of the tissue structure from the corresponding system;
generate an overlay frame comprising the location of the tissue structure; and
combine the overlay frame with a color image frame depicting the scene to indicate the location of the tissue structure within the scene.
14. The system of claim 13 , wherein the system generates a laser mapping exposure frame by sensing reflected electromagnetic radiation resulting from an emission by the mapping source, and wherein the controller is further configured to:
provide the laser mapping exposure frame to a corresponding system that determines a topology of the scene and/or dimensions of one or more objects within the scene;
provide the location of the tissue structure to the corresponding system; and
receive a topology and/or dimension of the tissue structure from the corresponding system.
15. The system of claim 1 , wherein the controller is configured to synchronize timing of the pulses of electromagnetic radiation during a blanking period of the image sensor, wherein the blanking period corresponds to a time between a readout of a last row of active pixels in the pixel array and a beginning of a next subsequent readout of active pixels in the pixel array.
16. The system of claim 1 , wherein the image sensor comprises a first image sensor and a second image sensor such that the image sensor can generate a three-dimensional image.
17. The system of claim 1 , wherein the spectral source of electromagnetic radiation is configured to emit electromagnetic radiation comprising one or more of:
electromagnetic radiation having a wavelength from about 513 nm to about 545 nm and electromagnetic radiation having a wavelength from about 900 nm to about 1000 nm; or
electromagnetic radiation having a wavelength from about 565 nm to about 585 nm and electromagnetic radiation having a wavelength from about 900 nm to about 1000 nm;
wherein sensing reflected electromagnetic radiation by the pixel array comprises generating a hyperspectral exposure frame based on the emission of the spectral source.
18. The system of claim 1 , wherein the image sensor senses reflected electromagnetic radiation from the visible and non-visible electromagnetic radiation for eliciting the spectral response from the tissue to generate a spectral exposure frame, and wherein the spectral exposure frame comprises information for determining one or more of an identification of a disease, a tissue condition, a biological process, a chemical process, or a type of tissue.
19. The system of claim 1 , wherein the controller is configured to independently actuate each of the plurality of sources of electromagnetic radiation in a pulse cycle pattern, and wherein the pulse cycle pattern is variable according to input from a user or feedback from the system.
20. The system of claim 1 , wherein the controller is further configured to generate a spectral datastore comprising a plurality of spectral responses, and wherein the system is configured to analyze the plurality of spectral responses to predict a classification of the tissue or a plurality of tissues within a scene.