Super resolution and color motion artifact correction in a pulsed color imaging system
The disclosure extends to methods, systems, and computer program products for producing an image in light deficient environments and associated structures, methods and features. The features of the systems and methods described herein may include providing improved resolution and color reproduction.
1. A digital imaging method for use with an endoscope in ambient light deficient environments comprising:
actuating an emitter to emit a pulse of a wavelength of electromagnetic radiation to cause illumination within the light deficient environment;
wherein said pulse is within a first wavelength range that comprises a first portion of electromagnetic spectrum;
pulsing said emitter at a predetermined interval;
sensing reflected electromagnetic radiation from said pulse with a pixel array;
wherein said pixel array is actuated at a sensing interval that corresponds to the pulse interval of said emitter;
detecting motion of objects being imaged;
increasing resolution for the pixel array by compensating for the detected motion by:
sensing luminescence of a plurality of neighboring pixels to gather luminance data,
bilinear interpolating the luminance data into a first upscaled data set,
bicubic interpolating the luminance data into a second upscaled data set, and
creating a baseline with no interpolation of the luminance data into a third upscaled data set; and
creating a stream of images by combining a plurality of sensed reflected electromagnetic energies into a frame.
2. The method of claim 1 , wherein said sensing process comprises sensing luminance and linear sums of luminance plus chrominance in adjacent images in a stream of images forming a video stream.
3. The method of claim 2 , further comprising indexing frames within the video stream with a rotating frame index.
4. The method of claim 3 , wherein the rotating frame index comprises four counts.
5. The method of claim 4 , further comprising reconstructing the video stream by combining luminance and chrominance data from a prior indexed frame.
6. The method of claim 4 , further comprising reconstructing the video stream by combining luminance and chrominance data from a prior indexed frame and following indexed frame.
7. The method of claim 4 , further comprising reconstructing a frame for luminance and two frames for chrominance.
8. The method of claim 4 , further comprising reconstructing the video stream by combining luminance and chrominance data from a plurality of prior indexed frames and a plurality of latter indexed frames for increased resolution and accuracy.
9. The method of claim 1 , wherein the first upscaled data set is used for block matching.
10. The method of claim 1 , wherein the second upscaled data set is used for fall back pixel data.
11. The method of claim 1 , wherein the third upscaled data set forms the baseline for the resolution-enhanced data set.
12. The method of claim 1 , further comprising segmenting data created by the pixel array into segments of pixels and nearest neighbors.
13. The method of claim 12 , further comprising shifting each segment of pixels in the x direction and comparing with a neighboring frame at the same resolution, in order to determine motion of an object being imaged in the x direction.
14. The method of claim 13 , further comprising shifting each segment of pixels in the x direction in sub-pixel increments and comparing to the first up-scaled data set for greater precision of motion detection in the x direction.
15. The method of claim 13 , further comprising shifting each segment of pixels in the y direction and comparing with a neighboring frame at the same resolution, in order to determine motion of an object being imaged in the y direction.
16. The method of claim 15 , further comprising shifting each segment of pixels in the y direction in sub-pixel increments and comparing to the first up-scaled data set for greater precision of motion detection in the y direction.
17. The method of claim 15 , further comprising determining the vector of the motion of the object by combining the x and y motion of the of the object being imaged.
18. The method of claim 17 , further comprising estimating motion to combine data from multiple luminance frames into a single, higher resolution luminance frame.
19. The method of claim 18 , wherein said process is repeated for every frame containing luminance data in a continuous sequence.