Surgical site measurement, and camera calibration using fiducial markers on surgical tools
Calibration of parameters of an endoscopic or laparoscopic camera is conducted while the camera is in use capturing images of a surgical procedure at a surgical site. The surgical procedure is performed using a surgical instrument marked with a fiducial pattern. A processor receives image data from images captured by the camera and uses machine vision to detect points of the pattern, and carries out an optimization to determine the 3D pose of the surgical instrument relative to the camera and the camera parameters. Once calibration parameters are determined, 3D measurements between measurement points in the body cavity may be estimated.
1. A system for measuring distances within a surgical site, comprising:
a camera positionable to capture image data corresponding to a surgical site within a body cavity;
at least one surgical instrument positionable in a body cavity, the surgical instrument including an elongate shaft and a fiducial calibration pattern on the elongate shaft;
at least one processor and at least one memory, the at least one memory storing instructions executable by said at least one processor to:
analyzing a plurality of frames of images of the fiducial calibration pattern captured by the camera with both the camera and the fiducial calibration pattern of the elongate shaft disposed within the body cavity, determining locations of fiducials in the calibration pattern in the plurality of frames, and, based on known geometry of the fiducials, estimating intrinsic camera parameters for the camera;
determining relative 3D positions of identified measurement points in the treatment site captured in images captured by the camera,
estimating a distance between the identified measurement points, and
generating output communicating the estimated distance to a user.
2. The system of claim 1 , wherein the distance is a straight line distance.
3. The system of claim 1 , wherein the distance is a geodesic distance following the topography of tissue surfaces between the measurement points.
4. The system of claim 1 , wherein the system includes a display, and the output includes generating an overlay displaying the estimated distance.
5. The system of claim 1 , wherein the system includes a display, and the instructions are further executable to generate an overlay marking the measurement points.
6. The system of claim 1 , wherein the instructions are further executable by said at least one processor to receive input from a user identifying the measurement points using a user input device.
7. The system of claim 1 , wherein the instructions are further executable by said at least one processor to identify the measurement points using computer vision.
8. The system of claim 6 , wherein the measurement points are points on or a predetermined distance from a part of said at least one surgical instrument disposed at the treatment site.
9. The system of claim 7 , wherein the measurement points are points on or a predetermined distance from a part of said at least one surgical instrument disposed at the treatment site.
10. The system of claim 6 , wherein the instructions are executable by said at least one processor to receive input from a user selecting a plane containing the measurement points, and to estimate or determine the distance between identified measurement points along the tissue surface where the plane intersects the tissue.
11. The system of claim 7 , wherein the instructions are executable by said at least one processor to receive input from a user selecting a plane containing the measurement points, and to estimate or determine the distance between identified measurement points along the tissue surface where the plane intersects the tissue.
12. The system of claim 1 , wherein the calibration pattern comprises a plurality of polyhedrons of known geometry.
13. The system of claim 1 , wherein the polyhedrons are rectangles.
14. The system of claim 1 , wherein the pattern is a checkerboard pattern.
15. The system of claim 1 , wherein the fiducial calibration pattern extends 360 degrees around a circumference of the shaft.
16. A method of measuring a distance between measurement points in a body cavity, comprising:
positioning a camera with its distal end at a surgical treatment site within a body cavity;
providing a surgical instrument having a fiducial calibration pattern thereon, and positioning the surgical instrument such that the fiducial calibration pattern is disposed within the body cavity;
with the distal end of the camera within the body cavity, using the camera to capture a plurality of images of the surgical instrument; and
determining locations of scene points in the fiducial calibration pattern in a plurality of frames of the images and, using the locations of the scene points and a known geometry of the fiducial calibration pattern, estimating intrinsic parameters of the camera;
estimating, from captured images of the surgical site, a three-dimensional distance between identified measurement points within the surgical site.
17. The method of claim 16 , wherein the fiducial calibration pattern is on the shaft of the instrument.
18. The method of claim 16 , further comprising the step of using the surgical instrument to treat tissue at the treatment site while estimating the camera parameters.
19. The method of claim 16 , further comprising the step of using the surgical instrument to diagnose a condition at the treatment site while estimating the camera parameters.
20. The method of claim 16 , further including generating output communicating the estimated distance to a user.