IP Library › Granted Patent US 12,439,017
Granted Patent B2
US 12,439,017 · App. 18/135,455 · Granted Oct 7, 2025

Method and system for enhanced image sensor timing

Inventors: David C. Shafer (Menlo Park, CA); Jeffrey M. DiCarlo (Austin, TX); Ian E. McDowall (Woodside, CA); Max J. Trejo (San Jose, CA)
Assignee: Intuitive Surgical Operations, Inc.
H04N13/296H04N13/239H04N13/257H04N23/11H04N23/13H04N23/73H04N23/84H04N25/135H04N25/7795
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,439,017
App. No.
18/135,455
Granted
Oct 7, 2025
Kind
B2
Abstract

An image capture device may include an image sensor comprising a first pixel cell and a second pixel cell adjacent to the first pixel cell, a plurality of individual alternative light filters configured to filter non-visible light, one individual alternative light filter of the plurality of individual alternative light filters covering both a first set of pixels of the first pixel cell and a second set of pixels of the second pixel cell, and a plurality of individual visible light color filters covering pixels of the first and second pixel cells, the pixels of the first and second pixel cells covered by the individual visible light color filters being different from the first and second sets of pixels covered by the individual alternative light filters.

Claims (58)

1. An image capture device comprising:

an image sensor comprising a first pixel cell and a second pixel cell adjacent to the first pixel cell;

a plurality of individual alternative light filters configured to filter non-visible light, one individual alternative light filter of the plurality of individual alternative light filters covering both a first set of pixels of the first pixel cell and a second set of pixels of the second pixel cell; and

a plurality of individual visible light color filters covering pixels of the first and second pixel cells, the pixels of the first and second pixel cells covered by the individual visible light color filters being different from the first and second sets of pixels covered by the individual alternative light filters.

2. The image capture device of claim 1 , wherein:

the first pixel cell comprises a first pixel, a second pixel, a third pixel, and a fourth pixel arranged in a first two-by-two array of pixels, the third and fourth pixels of the first pixel cell being on a bottom row of the first two-by-two array of pixels;

the second pixel cell comprises a first pixel, a second pixel, a third pixel, and a fourth pixel arranged in a second two-by-two array of pixels, the first and second pixels of the second pixel cell being on a top row of the second two-by-two array of pixels;

the first and second sets of pixels covered by the one individual alternative light filter includes only the third and fourth pixels of the first pixel cell and the first and second pixels of the second pixel cell; and

the pixels covered by the individual visible light color filters include only the first and second pixels of the first pixel cell and the third and fourth pixels of the second pixel cell.

3. The image capture device of claim 2 , wherein the individual visible light color filters comprise:

a red color filter covering the first pixel of the first pixel cell;

a first green color filter covering the second pixel of the first pixel cell;

a second green color filter covering the third pixel of the second pixel cell; and

a blue color filter covering the fourth pixel of the second pixel cell.

4. The image capture device of claim 1 , further comprising:

a first frame timer coupled to the image sensor to provide image capture timing signals to the image sensor to cause the image sensor to capture a first plurality of frames of pixel data at a first frame rate;

an additional image sensor; and

a second frame timer coupled to the additional image sensor to provide additional image capture timing signals to the additional image sensor to cause the additional image sensor to capture a second plurality of frames of pixel data at a second frame rate different than the first frame rate.

5. The image capture device of claim 4 :

the image capture timing signals provided by the first frame timer are configured to cause the image sensor to capture N frames of pixel data, where N is greater than one; and

the image capture timing signals provided by the second frame timer are configured to cause the additional image sensor to capture only a single frame of pixel data while the image sensor captures the N frames.

6. The image capture device of claim 4 :

the first frame timer being configured to expose pixels of the image sensor for a first exposure time;

the second frame timer being configured to expose pixels of the additional image sensor for a second exposure time; and

the first exposure time being different from the second exposure time.

7. The image capture device of claim 4 , the first frame timer being configured to simultaneously reset pixels in the first and second pixel cells covered by one of the plurality of individual visible light color filters.

8. The image capture device of claim 4 :

the first frame timer being configured to simultaneously read a first pixel in the first set of pixels and a second pixel of the second set of pixels.

9. The image capture device of claim 8 , wherein the image capture device is configured to bin the first pixel read by the first frame timer and the second pixel read by the first frame timer.

10. The image capture device of claim 1 :

the image sensor further comprising a plurality of visible light color filtered pixel cells interleaved with a plurality of alternative light filtered pixel cells.

11. The image capture device of claim 1 , wherein the image sensor comprises a Bayer color filter array.

12. A method comprising:

providing an image sensor comprising a first pixel cell and a second pixel cell adjacent to the first pixel cell;

causing a plurality of individual alternative light filters to filter non-visible light, one individual alternative light filter of the plurality of individual alternative light filters covering both a first set of pixels of the first pixel cell and a second set of pixels of the second pixel cell; and

causing a plurality of individual visible light color filters to filter visible light, the plurality of individual visible light color filters covering pixels of the first and second pixel cells, the pixels of the first and second pixel cells covered by the individual visible light color filters being different from the first and second sets of pixels covered by the individual alternative light filters.

13. The method of claim 12 , wherein:

the first pixel cell comprises a first pixel, a second pixel, a third pixel, and a fourth pixel arranged in a first two-by-two array of pixels, the third and fourth pixels of the first pixel cell being on a bottom row of the first two-by-two array of pixels;

the second pixel cell comprises a first pixel, a second pixel, a third pixel, and a fourth pixel arranged in a second two-by-two array of pixels, the first and second pixels of the second pixel cell being on a top row of the second two-by-two array of pixels;

the first and second sets of pixels covered by the one individual alternative light filter includes only the third and fourth pixels of the first pixel cell and the first and second pixels of the second pixel cell; and

the pixels covered by the individual visible light color filters include only the first and second pixels of the first pixel cell and the third and fourth pixels of the second pixel cell.

14. The method of claim 13 , wherein the individual visible light color filters comprise:

a red color filter covering the first pixel of the first pixel cell;

a first green color filter covering the second pixel of the first pixel cell;

a second green color filter covering the third pixel of the second pixel cell; and

a blue color filter covering the fourth pixel of the second pixel cell.

15. The method of claim 12 , further comprising:

causing a first frame timer coupled to the image sensor to provide image capture timing signals to the image sensor to cause the image sensor to capture a first plurality of frames of pixel data at a first frame rate; and

causing a second frame timer coupled to an additional image sensor to provide additional image capture timing signals to the additional image sensor to cause the additional image sensor to capture a second plurality of frames of pixel data at a second frame rate different than the first frame rate.

16. The method of claim 15 , wherein:

the image capture timing signals provided by the first frame timer are configured to cause the image sensor to capture N frames of pixel data, where N is greater than one; and

the image capture timing signals provided by the second frame timer are configured to cause the additional image sensor to capture only a single frame of pixel data while the image sensor captures the N frames.

17. The method of claim 15 , further comprising:

causing the first frame timer to expose pixels of the image sensor for a first exposure time;

causing the second frame timer to expose pixels of the additional image sensor for a second exposure time;

the first exposure time being different from the second exposure time.

18. The method of claim 15 , further comprising causing the first frame timer to simultaneously reset pixels in the first and second pixel cells covered by one of the plurality of individual visible light color filters.

19. The method of claim 15 , further comprising causing the first frame timer to simultaneously read a first pixel of the first pixel cell covered by one of the plurality of individual visible light color filters and a second pixel of the second pixel cell covered by one of the plurality of individual visible light color filters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 17, 2023
From: SHAFER, DAVID C.; DICARLO, JEFFREY M.; MCDOWALL, IAN E.; TREJO, MAX J.
To: INTUITIVE SURGICAL OPERATIONS, INC.
Reel/Frame 063345/0786 →
Continuity (3)
Continuation 17276061
Provisional Application 62732718 · Sep 18, 2018
Related Publication 20230254469A1 · Aug 10, 2023
References Cited (24)
US 11671581B2 · Shafer et al. · 2023 [cited by applicant]
US 20080211941A1 · Deever et al. · 2008 [cited by applicant]
US 20110273599A1 · Murata · 2011 [cited by applicant]
US 20140152773A1 · Ohba et al. · 2014 [cited by applicant]
US 20140198184A1 · Stein et al. · 2014 [cited by applicant]
US 20160182791A1 · Lewkow et al. · 2016 [cited by applicant]
US 20170026622A1 · Yoon et al. · 2017 [cited by applicant]
US 20170069098A1 · Johannesson · 2017 [cited by applicant]
US 20180188427A1 · Brueckner · 2018 [cited by examiner]
US 20190327394A1 · Ramirez Luna et al. · 2019 [cited by applicant]
US 20210360221A1 · Shafer et al. · 2021 [cited by applicant]
JP 2009296400A · 2009 [cited by applicant]
JP 2010162123A · 2010 [cited by applicant]
JP 2011523538A · 2011 [cited by applicant]
JP 2012244312A · 2012 [cited by applicant]
JP 2012253531A · 2012 [cited by applicant]
JP 2013121173A · 2013 [cited by applicant]
JP 2014238731A · 2014 [cited by applicant]
JP 2016143915A · 2016 [cited by applicant]
WO WO2015142800A1 · 2015 [cited by applicant]
International Preliminary Report on Patentability for Application No. PCT/US2019/051593, mailed on Apr. 1, 2021, 10 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2019/051593. mailed on Jan. 9, 2020, 14 pages. [cited by applicant]
Invitation to Pay Additional Fees and PartialInternational Search Report for PCT/US2019/051593, mailed Nov. 8, 2019, 13pages (ISRG07700/PCT). [cited by applicant]
Vertut, J, and Coiffet, P., “Robot Technology: Teleoperation and Robotics Evolution and Development,” English translation, Prentice-Hall, Inc., Inglewood Cliffs, NJ, USA 1986, vol. 3A, 332 pages. [cited by applicant]