Granted Patent B2

US 12,380,533 · App. 18/376,987 · Granted Aug 5, 2025

Inventor: Wenyi Zhao (Weston, FL)

Assignee: Intuitive Surgical Operations, Inc.

G06T3/4015H04N23/843H04N25/134A61B2576/00H04N23/555

Abstract

Claims (59)

1. A method comprising:

receiving a frame of raw pixels captured by an image capture sensor, a particular location in the frame of raw pixels including a captured color pixel value of a first color component of a first color space;

generating, for the particular location in the frame of raw pixels, a vertical color difference signal and a horizontal color difference signal using information from the frame of raw pixels;

determining that the particular location is missing a color pixel value of a second color component of the first color space;

generating, for the particular location in the frame of raw pixels, a reconstructed color pixel value of the second color component based on the vertical color difference signal and the horizontal color difference signal; and

transforming, for the particular location in the frame of raw pixels, the vertical color difference signal and the horizontal color difference signal of the particular location in the frame of raw pixels into chrominance pixel values in a second color space that is different from the first color space;

wherein transforming the vertical color difference signal and the horizontal color difference signal into the chrominance pixel values in the second color space comprises:

determining an average of the vertical color difference signal and the horizontal color difference signal; and

using the average of the vertical color difference signal and the horizontal color difference signal to determine the chrominance pixel values in the second color space.

2. The method of claim 1 , wherein for the particular location:

the captured color pixel value is a captured green pixel value;

the horizontal color difference signal is the captured green pixel value minus a horizontal estimated blue pixel value; and

the vertical color difference signal is the captured green pixel value minus a vertical estimated red pixel value.

3. The method of claim 1 , further comprising:

transforming, for the particular location in the frame of raw pixels, the captured color pixel value of the first color component to a brightness pixel value in the second color space.

4. The method of claim 3 , wherein transforming the captured color pixel value of the first color component to the brightness pixel value in the second color space is performed without using a color pixel value of a second color component of the first color space.

5. The method of claim 4 , wherein transforming the captured color pixel value of the first color component to the brightness pixel value in the second color space is performed without using a color pixel value of a third color component of the first color space.

6. The method of claim 1 , further comprising:

generating, for the particular location in the frame of raw pixels and using information from the frame of raw pixels, a vertical estimated color pixel value of the first color component of the first color space and a horizontal estimated color pixel value of the first color component of the first color space;

wherein generating the vertical color difference signal and the horizontal color difference signal for the particular location in the frame of pixels uses the vertical estimated color pixel value and the horizontal estimated color pixel value.

7. A system comprising:

a memory storing instructions; and

one or more processors communicatively coupled to the memory and configured to execute the instructions to perform a process comprising:

receiving a frame of raw pixels captured by an image capture sensor, a particular location in the frame of raw pixels including a captured color pixel value of a first color component of a first color space;

generating, for the particular location in the frame of raw pixels, a vertical color difference signal and a horizontal color difference signal using information from the frame of raw pixels;

determining that the particular location is missing a color pixel value of a second color component of the first color space;

generating, for the particular location in the frame of raw pixels, a reconstructed color pixel value of the second color component based on the vertical color difference signal and the horizontal color difference signal; and

transforming, for the particular location in the frame of raw pixels, the vertical color difference signal and the horizontal color difference signal of the particular location in the frame of raw pixels into chrominance pixel values in a second color space that is different from the first color space;

wherein transforming the vertical color difference signal and the horizontal color difference signal into the chrominance pixel values in the second color space comprises:

determining an average of the vertical color difference signal and the horizontal color difference signal; and

using the average of the vertical color difference signal and the horizontal color difference signal to determine the chrominance pixel values in the second color space.

8. The system of claim 7 , wherein for the particular location:

the captured color pixel value is a captured green pixel value;

the horizontal color difference signal is the captured green pixel value minus a horizontal estimated blue pixel value; and

the vertical color difference signal is the captured green pixel value minus a vertical estimated red pixel value.

9. The system of claim 7 , wherein the process further comprises:

transforming, for the particular location in the frame of raw pixels, the captured color pixel value of the first color component to a brightness pixel value in the second color space.

10. The system of claim 9 , wherein transforming the captured color pixel value of the first color component to the brightness pixel value in the second color space is performed without using a color pixel value of a second color component of the first color space.

11. The system of claim 10 , wherein transforming the captured color pixel value of the first color component to the brightness pixel value in the second color space is performed without using a color pixel value of a third color component of the first color space.

12. The system of claim 7 , wherein the process further comprises:

generating, for the particular location in the frame of raw pixels and using information from the frame of raw pixels, a vertical estimated color pixel value of the first color component of the first color space and a horizontal estimated color pixel value of the first color component of the first color space;

wherein generating the vertical color difference signal and the horizontal color difference signal for the particular location in the frame of pixels uses the vertical estimated color pixel value and the horizontal estimated color pixel value.

13. A non-transitory computer-readable medium storing instructions executable by a processor to perform a process comprising:

receiving a frame of raw pixels captured by an image capture sensor, a particular location in the frame of raw pixels including a captured color pixel value of a first color component of a first color space;

generating, for the particular location in the frame of raw pixels, a vertical color difference signal and a horizontal color difference signal using information from the frame of raw pixels;

determining that the particular location is missing a color pixel value of a second color component of the first color space;

generating, for the particular location in the frame of raw pixels, a reconstructed color pixel value of the second color component based on the vertical color difference signal and the horizontal color difference signal; and

transforming, for the particular location in the frame of raw pixels, the vertical color difference signal and the horizontal color difference signal of the particular location in the frame of raw pixels into chrominance pixel values in a second color space that is different from the first color space;

wherein transforming the vertical color difference signal and the horizontal color difference signal into the chrominance pixel values in the second color space comprises:

determining an average of the vertical color difference signal and the horizontal color difference signal; and

using the average of the vertical color difference signal and the horizontal color difference signal to determine the chrominance pixel values in the second color space.

14. The non-transitory computer-readable medium of claim 13 , wherein for the particular location:

the captured color pixel value is a captured green pixel value;

the horizontal color difference signal is the captured green pixel value minus a horizontal estimated blue pixel value; and

the vertical color difference signal is the captured green pixel value minus a vertical estimated red pixel value.

15. The non-transitory computer-readable medium of claim 13 , wherein the process further comprises:

transforming, for the particular location in the frame of raw pixels, the captured color pixel value of the first color component to a brightness pixel value in the second color space.

16. The non-transitory computer-readable medium of claim 15 , wherein transforming the captured color pixel value of the first color component to the brightness pixel value in the second color space is performed without using a color pixel value of a second color component of the first color space.

17. The non-transitory computer-readable medium of claim 16 , wherein transforming the captured color pixel value of the first color component to the brightness pixel value in the second color space is performed without using a color pixel value of a third color component of the first color space.

Assignments (1)

Continuity (6)

References Cited (33)