Systems and methods for generating three-dimensional measurements using endoscopic video data
Presented herein are systems and methods for performing three-dimensional measurements of a surgical space using two-dimensional endoscopic images. According to an aspect, video data taken from an endoscopic imaging device can be used to generate a three-dimensional model of the surgical space represented by the video data. In one or more examples, two-dimensional images from the video data can be used to generate a three-dimensional model of the surgical space. In one or more examples, the one or more two-dimensional images of the surgical space can include a fiducial marker as part of the image. Using both the depth information and a size reference provided by the fiducial marker, the systems and methods herein can generate a three-dimensional model of the surgical space. The generated three-dimensional model can then be used to perform a variety of three-dimensional measurements in a surgical cavity in an accurate and efficient manner.
1 . A method for measuring distances using endoscopic images, the method comprising:
receiving, at one or more processors, video data from an endoscopic imaging device configured to image an internal area of a patient, wherein the video data comprises one or more two-dimensional images that capture one or more fiducials;
identifying, by the one or more processors, a first point of the internal area of the patient in the one or more two-dimensional images;
identifying, by the one or more processors, a second point of the internal area of the patient in the one or more two-dimensional images; and
determining, by the one or more processors and based on the one or more fiducials, a measurement between the first point and the second point that accounts for curvature of the internal area of the patient.
2 . The method of claim 1 , wherein determining the measurement comprises using the one or more fiducials for scale.
3 . The method of claim 1 , wherein determining the measurement comprises determining depth information based on the one or more two-dimensional images.
4 . The method of claim 3 , wherein the depth information is determined from motion between multiple images of the one or more images.
5 . The method of claim 3 , wherein the depth information is determined using a machine learning model on a single image of the one or more images.
6 . The method of claim 1 , wherein the measurement is determined based on a three-dimensional model of the internal area of the patient.
7 . The method of claim 6 , wherein determining the measurement comprises generating the three-dimensional model based on the one or more two-dimensional images.
8 . The method of claim 1 , wherein the one or more fiducials comprises an object of a predetermined size.
9 . The method of claim 1 , wherein the one or more fiducials comprises a fiducial marker disposed on a surface of a surgical instrument.
10 . The method of claim 1 , wherein identifying the first point of the internal area of the patient in the one or more two-dimensional images comprises identifying the first point based on a received first user input, and wherein identifying the second point of the internal area of the patient in the one or more two-dimensional images comprises identifying the second point based on a received second user input.
11 . A system for measuring distances using endoscopic images, the system comprising:
a memory; and
one or more processors,
wherein the memory stores one or more programs that when executed by the one or more processors, cause the one or more processors to:
receive video data from an endoscopic imaging device configured to image an internal area of a patient, wherein the video data comprises one or more two-dimensional images that capture one or more fiducials;
identify a first point of the internal area of the patient in the one or more two-dimensional images;
identify a second point of the internal area of the patient in the one or more two-dimensional images; and
determine, based on the one or more fiducials, a measurement between the first point and the second point that accounts for curvature of the internal area of the patient.
12 . The system of claim 11 , wherein determining the measurement comprises using the one or more fiducials for scale.
13 . The system of claim 11 , wherein determining the measurement comprises determining depth information based on the one or more two-dimensional images.
14 . The system of claim 13 , wherein the depth information is determined from motion between multiple images of the one or more images.
15 . The system of claim 13 , wherein the depth information is determined using a machine learning model on a single image of the one or more images.
16 . The system of claim 11 , wherein the measurement is determined based on a three-dimensional model of the internal area of the patient.
17 . The system of claim 16 , wherein determining the measurement comprises generating the three-dimensional model based on the one or more two-dimensional images.
18 . The system of claim 11 , wherein the one or more fiducials comprises an object of a predetermined size.
19 . The system of claim 11 , wherein the one or more fiducials comprises a fiducial marker disposed on a surface of a surgical instrument.
20 . The system of claim 11 , wherein identifying the first point of the internal area of the patient in the one or more two-dimensional images comprises identifying the first point based on a received first user input, and wherein identifying the second point of the internal area of the patient in the one or more two-dimensional images comprises identifying the second point based on a received second user input.
21 . A non-transitory computer readable storage medium storing one or more programs for execution by one or more processors of a computing system, the one or more programs including instructions that, when executed by the computing system, cause the computing system to:
receive video data from an endoscopic imaging device configured to image an internal area of a patient, wherein the video data comprises one or more two-dimensional images that capture one or more fiducials;
identify a first point of the internal area of the patient in the one or more two-dimensional images;
identify a second point of the internal area of the patient in the one or more two-dimensional images; and
determine, based on the one or more fiducials, a measurement between the first point and the second point that accounts for curvature of the internal area of the patient.
22 . The non-transitory computer readable storage medium of claim 21 , wherein determining the measurement comprises using the one or more fiducials for scale.
23 . The non-transitory computer readable storage medium of claim 21 , wherein identifying the first point of the internal area of the patient in the one or more two-dimensional images comprises identifying the first point based on a received first user input, and wherein identifying the second point of the internal area of the patient in the one or more two-dimensional images comprises identifying the second point based on a received second user input.