Systems and methods for computer-assisted shape measurements in video
Various embodiments of the invention provide systems and methods to assist or guide an arthroscopic surgery (e.g., surgery of the shoulder, knee, hip or spine) or other surgical procedure by providing measurements intraoperatively. The systems and methods comprise steps of receiving a video stream from an arthroscopic imaging device; receiving one or more sets of coordinates of one or more points, paths, or area; calculating a length, a surface area, or a volume measurement based at least in part on the one or more sets of coordinates; overlaying the selected one or more sets of coordinates and the measurement on the video stream or on an expanded view of a region of surgery; and displaying the overlay on one or more display devices intraoperatively so as to be used by an operator during the arthroscopic procedure.
1 . A system for guiding a minimally invasive procedure, the system comprising one or more computer processors and one or more non-transitory computer-readable storage media storing instructions that are operable, when executed by said one or more computer processors, to cause said one or more computer processors to perform operations comprising:
receiving a video stream captured by an imaging device;
processing said video stream using processing modules including: a tool recognition module, a tooltip reference sizing module, an anatomy recognition module, and a tool tracking module, wherein said processing comprises:
identifying a surgical tool and one or more elements in said video stream, wherein said one or more elements comprises one or more of an anatomical structure, an operational procedure or action, or a pathology; and
labeling said identified one or more elements in said video stream;
receiving an indication from an operator to select one or more of a plurality of points or an area in said video stream;
calculating one or more measurements for said selected area in said video stream; wherein said one or more measurements comprise one or more of a length, a surface area, a circumference, or a volume, further wherein said tooltip reference sizing module estimates a size of tooltip of said identified surgical tool using a trained machine learning algorithm, and wherein a pathology sizing module estimates a size of said identified pathology based at least partially on said estimated size of tooltip; and
displaying said measurements on a displaying device intraoperatively to be used by an operator during said arthroscopic procedure.
2 . The system of claim 1 , wherein said processing modules further include one or more of: a video stream decomposition module, a gesture recognition module, a landmark establishment module, a distance aggregator module, a video blend module, a pathology detection module, a pathology sizing module, or a radiology imaging module.
3 . The system of claim 2 , wherein said processing modules each comprise one or more trained machine learning algorithms.
4 . The system of claim 3 , wherein each trained machine learning algorithm is a neural network.
5 . The system of claim 2 , wherein said video stream decomposition module decomposes said video stream into a series of images.
6 . The system of claim 5 , wherein said series of images are stored in a memory device.
7 . The system of claim 2 , wherein said radiological imaging module comprises: a radiology ingest module, a three-dimensional image construction module, a visual field mapping module, a real-time volume computation module.
8 . The system of claim 7 , wherein said radiology ingest module interfaces with an external repository to import a subject's radiological imagery.
9 . The system of claim 8 , wherein said radiological imagery comprises one or more of magnetic resonance imaging (MRI), computed tomography (CT) scanning, ultrasound, or a combination thereof.
10 . The system of claim 8 , said radiological imagery includes an image of a landmark.
11 . The system of claim 8 , said radiological imagery comprises volumetric data.
12 . The system of claim 11 , a three-dimensional image reconstruction module generates a three-dimensional object from said volumetric radiological imagery.
13 . The system of claim 12 , said three-dimensional image reconstruction module further generates coordinates to map said landmark onto said three-dimensional object.
14 . The system of claim 1 , wherein said labeling said video stream further comprises displaying a shape outline surrounding said one or more elements.
15 . The system of claim 1 , wherein said operator uses a standard surgical probe to select said area in said video stream.
16 . The system of claim 1 , wherein calculating one or more measurements comprises calculating a distance between a first point and a second point.
17 . The system of claim 16 , wherein at least one of said first point or said second point moves outside of said field of view.
18 . The system of claim 1 , wherein calculating said one or more measurements for said selected area comprises performing a volume calculation, wherein said volume calculation comprises:
receiving a three dimensional radiology image of a subject;
receiving an indication from an operator to select at least one element from said one or more elements in said video stream; and
calculating a volume of said selected area based at least partially on the selected at least one element on the three dimensional radiological image.
19 . The system of claim 18 , wherein receiving the indication of the operator to select said at least one element from said or more elements in said video stream comprises receiving an outline of a selected area from the operator.
20 . The system of claim 1 , wherein said operations further comprise:
generating an overlay video stream by blending said labels or said measurements with said received video stream; and
displaying said overlaid video stream on a displaying device.