Granted Patent B2

US 12,231,786 · App. 18/369,304 · Granted Feb 18, 2025

Inventors: Wenyi Zhao (Weston, FL); Jeffrey M. DiCarlo (Austin, TX); Ian E. McDowall (Woodside, CA)

Assignee: Intuitive Surgical Operations, Inc.

H04N25/134A61B1/00006A61B1/00009A61B1/0005A61B1/00193A61B1/043A61B1/05A61B1/06A61B1/0638A61B1/0646G06T3/4015H04N5/265H04N13/257H04N23/10H04N23/45H04N23/56H04N23/57H04N23/555

Abstract

Claims (64)

1. A method comprising:

causing an illuminator to simultaneously illuminate tissue with a first visible color illumination component, a second visible color illumination component, and a fluorescence excitation illumination component;

causing a display unit to display, based on the illumination, a display scene that includes:

a reduced color scene component corresponding to the first and second visible color illumination components, and

a highlighted scene component corresponding to the fluorescence excitation illumination component; and

receiving a single frame, the single frame comprising:

a first set of pixel data that includes a fluorescence scene component associated with the fluorescence excitation illumination component,

a second set of pixel data that includes a combination of a first visible color component scene and the fluorescence scene component, the first visible color component scene associated with the first visible color illumination component, and

a third set of pixel data that includes a combination of a second visible color component scene and the fluorescence scene component, the second visible color component scene associated with the second visible color illumination component;

wherein the causing the display unit to display the display scene is based on the first, second, and third sets of pixel data.

2. The method of claim 1 , further comprising capturing the single frame using a single-chip image sensor having a color filter array.

3. The method of claim 2 , wherein the capturing the single frame comprises:

capturing the first set of pixel data with a first set of pixels of the single-chip image capture sensor, the first set of pixels associated with a first color filter included in the color filter array; and

capturing the second set of pixel data with a second set of pixels of the single-chip image capture sensor, the second set of pixels associated with a second color filter included in the color filter array.

4. The method of claim 3 , wherein:

the first color filter is configured to allow light having a first color and fluorescence that results from the fluorescence excitation illumination component to pass through the first color filter to the first set of pixels; and

the second color filter is configured to allow light having a second color and the fluorescence to pass through the second color filter to the second set of pixels.

5. The method of claim 3 , wherein the second set of pixels are covered by a green color filter included in the color filter array.

6. The method of claim 3 , wherein the second set of pixels are covered by a blue color filter included in the color filter array.

7. The method of claim 1 , further comprising:

receiving, in a first color channel, the first set of pixel data; and

demosaicing the first set of pixel data to obtain a first set of image pixel data.

8. The method of claim 7 , further comprising:

receiving, in a second color channel, the second set of pixel data; and

demosaicing the second set of pixel data to obtain a second set of image pixel data.

9. The method of claim 8 , further comprising:

determining, based on the first and second sets of image pixel data, a display fluorescence scene component and a display visible scene component;

generating the display scene based on the display fluorescence scene component and the display visible scene component.

10. The method of claim 9 , wherein the display fluorescence scene component comprises a first linear weighted combination of the first set of image pixel data and the second set of image pixel data.

11. The method of claim 10 , wherein the display visible scene component comprises a second linear weighted combination of the first set of image pixel data and the second set of image pixel data.

12. The method of claim 1 , further comprising:

causing the illuminator to change from illuminating the tissue with the first visible color illumination component, the second visible color illumination component, and the fluorescence excitation illumination component to illuminating the tissue with full color illumination that includes the first visible color illumination component, the second visible color illumination component, and a third visible color component; and

causing, based on the illuminator changing to illuminating the tissue with the full color illumination, the display unit to display a full visible color scene of the tissue in place of the display scene.

13. A system comprising:

a memory storing instructions; and

one or more processors communicatively coupled to the memory and configured to execute the instructions to perform a process comprising:

causing an illuminator to simultaneously illuminate tissue with a first visible color illumination component, a second visible color illumination component, and a fluorescence excitation illumination component;

causing a display unit to display, based on the illumination, a display scene that includes:

a reduced color scene component corresponding to the first and second visible color illumination components, and

a highlighted scene component corresponding to the fluorescence excitation illumination component; and

receiving a single frame, the single frame comprising:

a first set of pixel data that includes a fluorescence scene component associated with the fluorescence excitation illumination component,

a second set of pixel data that includes a combination of a first visible color component scene and the fluorescence scene component, the first visible color component scene associated with the first visible color illumination component, and

a third set of pixel data that includes a combination of a second visible color component scene and the fluorescence scene component, the second visible color component scene associated with the second visible color illumination component;

wherein the causing the display unit to display the display scene is based on the first, second, and third sets of pixel data.

14. The system of claim 13 , further comprising capturing the single frame using a single-chip image sensor having a color filter array.

15. The system of claim 14 , wherein the capturing the single frame comprises:

capturing the first set of pixel data with a first set of pixels of the single-chip image capture sensor, the first set of pixels associated with a first color filter included in the color filter array; and

capturing the second set of pixel data with a second set of pixels of the single-chip image capture sensor, the second set of pixels associated with a second color filter included in the color filter array.

16. The system of claim 15 , wherein:

the first color filter is configured to allow light having a first color and fluorescence that results from the fluorescence excitation illumination component to pass through the first color filter to the first set of pixels; and

the second color filter is configured to allow light having a second color and the fluorescence to pass through the second color filter to the second set of pixels.

17. The system of claim 15 , wherein the second set of pixels are covered by a green color filter included in the color filter array.

18. The system of claim 13 , wherein the process further comprises:

causing the illuminator to change from illuminating the tissue with the first visible color illumination component, the second visible color illumination component, and the fluorescence excitation illumination component to illuminating the tissue with full color illumination that includes the first visible color illumination component, the second visible color illumination component, and a third visible color component; and

causing, based on the illuminator changing to illuminating the tissue with the full color illumination, the display unit to display a full visible color scene of the tissue in place of the display scene.

19. The system of claim 13 , wherein the process further comprises:

receiving, in a first color channel, the first set of pixel data;

demosaicing the first set of pixel data to obtain a first set of image pixel data;

receiving, in a second color channel, the second set of pixel data; and

demosaicing the second set of pixel data to obtain a second set of image pixel data.

20. The system of claim 19 , wherein the process further comprises:

determining, based on the first and second sets of image pixel data, a display fluorescence scene component and a display visible scene component; and

generating the display scene based on the display fluorescence scene component and the display visible scene component.

Assignments (1)

Continuity (10)

References Cited (42)