IP Library › Granted Patent US 10,165,195
Granted Patent B2
US 10,165,195 · App. 15/483,934 · Granted Dec 25, 2018

Wide dynamic range using monochromatic sensor

Inventors: Laurent Blanquart (Westlake Village, CA); John Richardson (Westlake Village, CA)
Assignee: DePuy Synthes Products, Inc.
H04N5/2355A61B1/045A61B1/06H04N5/2256H04N5/2354H04N5/35554H04N5/37457H04N9/045H04N9/07H04N13/239H04N5/35536H04N5/35563H04N2209/042
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,165,195
App. No.
15/483,934
Granted
Dec 25, 2018
Kind
B2
Abstract

The disclosure extends to methods, systems, and computer program products for widening dynamic range within an image in a light deficient environment.

Claims (54)

1. A method for imaging in a light deficient environment comprising:

emitting four electromagnetic radiation pulses of three different wavelengths into a light deficient environment during a blanking period during frame capture of an image sensor, wherein the image sensor comprises a plurality of pixels, and wherein the image sensor comprises a monochromatic image sensor;

capturing a plurality of frames in the light deficient environment using the image sensor, each one of the plurality of frames being radiated by one of the four electromagnetic radiation pulses of the three different wavelengths, wherein capturing at least one frame of the plurality frames comprises controlling a first subset of pixels during the at least one frame for a long exposure and controlling a second subset of pixels during the at least one frame for a short exposure, wherein the at least one frame for the long exposure and the at least one frame for the short exposure are two of the plurality of frames that are exposed to the same wavelength of electromagnetic radiation; and

creating a single display frame from the plurality of frames.

2. The method of claim 1 , further comprising:

creating a stream of display frames by repeating the emitting electromagnetic radiation into the light deficient environment, the capturing the plurality of frames, and the creating the single display frame from the plurality of frames; and

providing the stream of display frames as a video stream.

3. The method of claim 1 , wherein each of the pixels comprises a transfer gate transistor (TX).

4. The method of claim 3 , wherein each TX in a subgroup of pixels is electrically connected together by a common TX line.

5. The method of claim 3 , wherein the TX line provides global operation of transfer gate transistors for the subgroup of pixels.

6. The method of claim 1 , wherein two or more pixels of the image sensor share a floating diffusion in a pixel share, wherein a first subgroup of pixels in the first subset is connected by a first TX line and a second subgroup of pixels in the second subset is connected by a second TX line.

7. The method of claim 6 , wherein the two or more pixels are within a same row of pixels of the image sensor.

8. The method of claim 6 , wherein global operation of the first TX line and the second TX line is performed during the blanking period.

9. The method of claim 6 , wherein controlling the first subset of pixels during the at least one frame for the long exposure and controlling the second subset of pixels during the at least one frame for the short exposure comprises actuating the first TX line at a different time than the second TX line.

10. The method of claim 1 , wherein the first subset of pixels and the second subset of pixels are arranged in a checkerboard pattern.

11. The method of claim 1 , further comprising capturing at least a second frame during which the first subset of pixels is exposed for the short exposure and the second subset of pixels are exposed for the long exposure, wherein the long exposure comprises a longer length of time than the short exposure.

12. The method of claim 1 , wherein emitting the electromagnetic radiation into the light deficient environment comprises controlling the emitter to pulse electromagnetic energy at a plurality of exposures corresponding to subsets of different pixels of the image sensor.

13. A system for digital imaging in a light deficient environment:

an emitter for providing illumination using electromagnetic pulses of a plurality of different wavelengths;

a monochromatic image sensor comprising a plurality of pixels that are sensitive to the electromagnetic pulses; and

a controller that causes the system to:

emit four electromagnetic radiation pulses of three different wavelengths into a light deficient environment during a blanking period during frame capture of the image sensor;

capture a plurality of frames in the light deficient environment using the image sensor, each one of the plurality of frames being radiated by one of the four electromagnetic pulses of the three different wavelengths, wherein capturing at least one frame of the repeating frames comprises controlling a first subset of pixels during the at least one frame for a long exposure and controlling a second subset of pixels during the at least one frame for a short exposure, wherein the at least one frame for the long exposure and the at least one frame for the short exposure are two of the plurality of frames that are exposed to the same wavelength of electromagnetic radiation; and

create a single display frame from the plurality of frames.

14. The system of claim 13 , wherein the controller causes the system to:

create a stream of display frames by repeating the emitting electromagnetic radiation into the light deficient environment, the capturing the plurality of frames, and the creating the single display frame from the plurality of frames; and

provide the stream of display frames as a video stream.

15. The system of claim 13 , wherein each of the pixels of the image sensor comprises a transfer gate transistor (TX).

16. The system of claim 15 , wherein each TX in a subgroup of pixels is electrically connected together by a common TX line.

17. The system of claim 15 , wherein the TX line provides global operation of transfer gate transistors for the subgroup of pixels.

18. The system of claim 13 , wherein two or more pixels of the image sensor share a floating diffusion in a pixel share, wherein a first subgroup of pixels in the first subset is connected by a first TX line and a second subgroup of pixels in the second subset is connected by a second TX line.

19. The system of claim 18 , wherein the two or more pixels are within a same row of pixels of the image sensor.

20. The system of claim 18 , wherein global operation of the first TX line and the second TX line is performed during the blanking period.

21. The system of claim 18 , wherein the controller controls the first subset of pixels during the at least one frame for the long exposure and controls the second subset of pixels during the at least one frame for the short exposure by actuating the first TX line at a different time than the second TX line.

22. The system of claim 13 , wherein the first subset of pixels and the second subset of pixels are arranged in a checkerboard pattern in the image sensor.

23. The system of claim 13 , wherein the controller further controls the system to capture at least a second frame during which the first subset of pixels is exposed for the short exposure and the second subset of pixels are exposed for the long exposure, wherein the long exposure comprises a longer length of time than the short exposure.

24. The system of claim 13 , wherein the controller causes the system to emit the electromagnetic radiation into the light deficient environment by controlling the emitter to pulse electromagnetic energy at a plurality of exposures corresponding to subsets of different pixels of the image sensor.

25. Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to:

emit four electromagnetic radiation pulses of three different wavelengths into a light deficient environment during a blanking period during frame capture of an image sensor, wherein the image sensor comprises a plurality of pixels, and wherein the image sensor comprises a monochromatic image sensor;

capture a plurality of frames in the light deficient environment using the image sensor, each one of the plurality of frames being radiated by one of the four electromagnetic radiation pulses of the three different wavelengths, wherein capturing at least one frame of the repeating frames comprises controlling a first subset of pixels during the at least one frame for a long exposure and controlling a second subset of pixels during the at least one frame for a short exposure, wherein the at least one frame for the long exposure and the at least one frame for the short exposure are two of the plurality of frames that are exposed to the same wavelength of electromagnetic radiation; and

create a single display frame from the plurality of frames.

26. The computer readable storage media of claim 25 , wherein the instructions further cause the one or more processors to:

create a stream of display frames by repeating the emitting electromagnetic radiation into the light deficient environment, the capturing the plurality of frames, and the creating the single display frame from the plurality of frames; and

provide the stream of display frames as a video stream.

27. The computer readable storage media of claim 25 , wherein each of the pixels of the image sensor comprises a transfer gate transistor (TX).

28. The computer readable storage media of claim 27 , wherein each TX in a subgroup of pixels is electrically connected together by a common TX line.

29. The computer readable storage media of claim 27 , wherein the TX line provides global operation of transfer gate transistors for the subset of pixels.

30. The computer readable storage media of claim 25 , wherein two or more pixels of the image sensor share a floating diffusion in a pixel share, wherein a first subgroup of pixels in the first subset is connected by a first TX line and a second subgroup of pixels in the second subset is connected by a second TX line.

31. The computer readable storage media of claim 30 , wherein the two or more pixels are within a same row of pixels of the image sensor.

32. The computer readable storage media of claim 30 , wherein the instructions cause global operation of the first TX line and the second TX line to be performed during the blanking period.

33. The computer readable storage media of claim 30 , wherein the instructions cause the one or more processors to control the first subset of pixels during the at least one frame for the long exposure and control the second subset of pixels during the at least one frame for the short exposure by actuating the first TX line at a different time than the second TX line.

34. The computer readable storage media of claim 25 , wherein the first subset of pixels and the second subset of pixels are arranged in a checkerboard pattern in the image sensor.

35. The computer readable storage media of claim 25 , wherein the instructions further cause the one or more processors to capture at least a second frame during which the first subset of pixels is exposed for the short exposure and the second subset of pixels are exposed for the long exposure, wherein the long exposure comprises a longer length of time than the short exposure.

36. The computer readable storage media of claim 25 , wherein the instructions further cause the one or more processors to emit the electromagnetic radiation into the light deficient environment by controlling the emitter to pulse electromagnetic energy at a plurality of exposures corresponding to subsets of different pixels of the image sensor.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 11, 2017
From: BLANQUART, LAURENT; RICHARDSON, JOHN
To: DEPUY SYNTHES PRODUCTS, INC.
Reel/Frame 041966/0198 →
Continuity (6)
Continuation 15362512 · Nov 28, 2016
Division 13952564 · Jul 26, 2013
Provisional Application 61790719 · Mar 15, 2013
Provisional Application 61790487 · Mar 15, 2013
Provisional Application 61676289 · Jul 26, 2012
Related Publication 20170214841A1 · Jul 27, 2017
Cited By (4)
US 12,238,265 US 12,316,965 US 12,470,831 US 12,647,691