Granted Patent B2

US 12,231,784 · App. 18/146,975 · Granted Feb 18, 2025

Inventors: John Richardson (Calabasas, CA); Laurent Blanquart (Westlake Village, CA)

Assignee: DePuy Synthes Products, Inc.

H04N23/951G06T3/4007G06T3/4053H04N23/12H04N23/56H04N23/60H04N23/74H04N23/843H04N13/239H04N23/555

Abstract

Claims (34)

1. A digital imaging method comprising:

actuating an emitter to emit a pulse of a wavelength of electromagnetic radiation to cause illumination within the light deficient environment;

sensing reflected electromagnetic radiation from each of the plurality of pulses of electromagnetic radiation with a pixel array;

performing sensor corrections to compensate for detected motion;

generating a plurality of exposure frames from the sensed electromagnetic radiation, wherein the plurality of exposure frames comprises a luminance frame and a chrominance frame;

indexing the generated plurality of exposure frames;

determining a motion estimate based on the luminance frame; and

modulating the chrominance frame based on the motion estimate.

2. The method of claim 1 , wherein performing sensor corrections to compensate for detected motion comprises increasing resolution of the pixel array by:

using bilinear interpolation on the luminance data to create a first upscaled data set; and

using bicubic interpolation on the luminance data to create a second upscaled data set.

3. The method of claim 2 , wherein performing sensor corrections to compensate for detected motion further comprises increasing resolution of the pixel array by using no interpolation to create a third upscaled data set, wherein the third upscaled data set forms the baseline for the resolution-enhanced data set.

4. The method of claim 3 , wherein detecting motion based on the luminance frame comprises:

determining a quantity of blank pixels within the third upscaled data set;

calculating a minimum sum of absolute differences in position of the blank pixels relative to an originating block of pixels; and

storing the sum as a motion estimate metric.

5. The method of claim 1 , further comprising sensing luminescence to gather luminance data comprises sensing luminance and linear sums of luminance plus chrominance in adjacent images in a stream of images forming a video stream.

6. The method of claim 5 , wherein indexing the generated plurality of exposure frames comprises indexing the plurality of exposure frames using a rotating frame index, and wherein the method further comprises reconstructing the video stream by combining luminance and chrominance data from a prior indexed frame.

7. The method of claim 6 , further comprising reconstructing the video stream by combining luminance and chrominance data from a prior indexed frame and following indexed frame.

8. The method of claim 1 , wherein indexing the generated plurality of exposure frames comprises constructing a first first-in-first-out (FIFO) queue for luminance data and a second first-in-first-out (FIFO) queue for chrominance data.

9. The method of claim 8 , wherein the first FIFO queue for luminance data comprises a minimum depth of 3 frames.

10. The method of claim 8 , wherein the FIFO queue for chrominance data comprises minimum depth of 2 frames.

11. The method of claim 8 , wherein the first FIFO queue for luminance data comprises a plurality of luminance frames, wherein at least one of the plurality of luminance frames is an object frame positioned centrally in the first FIFO queue for luminance data.

12. The method of claim 11 , wherein motion vectors are determined for two luminance frames of the plurality of luminance frames, and wherein the two luminance frames are flanking the object frame.

13. The method of claim 12 , wherein an intermediate chrominance frame is associated with one of the two luminance frames, and wherein the motion vectors are utilized to predict motion within the chrominance frame.

14. The method of claim 1 , further comprising:

segmenting data created by the pixel array into segments of pixels and nearest neighbors; and

shifting each segment of pixels in the x direction and comparing with a neighboring frame at the same resolution, to determine motion of an object being imaged in the x direction.

15. The method of claim 14 , further comprising shifting each segment of pixels in the x direction in sub-pixel increments and comparing to the first up-scaled data set for greater precision of motion detection in the x direction.

16. The method of claim 14 , further comprising shifting each segment of pixels in the y direction and comparing with a neighboring frame at the same resolution, to determine motion of an object being imaged in the y direction.

17. The method of claim 16 , further comprising shifting each segment of pixels in the y direction in sub-pixel increments and comparing to the first up-scaled data set for greater precision of motion detection in the y direction.

18. The method of claim 16 , further comprising determining the vector of the motion of the object by combining the x and y motion of the of the object being imaged.

19. The method of claim 18 , further comprising estimating motion to combine data from multiple luminance frames into a single, higher resolution luminance frame.

20. The method of claim 8 , wherein modulating the chrominance frame comprises utilizing the motion estimate metric to predict motion within the chrominance frame relative to the luminance frame.

Assignments (1)

Continuity (8)

References Cited (400)