METHODS AND SYSTEMS FOR EXTRACTING MEDICAL IMAGES
The present disclosure relates generally to medical imaging, and more specifically to extracting a subset of images from a series of images (e.g., surgical video feeds) for training machine-learning models and/or conducting various downstream analyses. The system can hash image data for each image of a series of video images of the surgery to obtain a series of hash values; calculate a plurality of difference values for the series of hash values, each of the plurality of difference values indicative of a difference between two consecutive hash values in the series of hash values; generate a plurality of image clusters by clustering the plurality of distance values; select one or more image clusters from the plurality of image clusters; and produce a subset of surgical images from the series of video images using the selected one or more image clusters from the plurality of image clusters.
1 . A method for obtaining a subset of surgical images from a series of video images of a surgery, comprising:
hashing image data for each image of the series of video images of the surgery to obtain a series of hash values;
calculating a plurality of difference values for the series of hash values, each of the plurality of difference values indicative of a difference between two consecutive hash values in the series of hash values;
generating a plurality of image clusters by clustering the plurality of difference values;
selecting one or more image clusters from the plurality of image clusters; and
producing the subset of surgical images from the series of video images using the selected one or more image clusters from the plurality of image clusters.
2 . The method of claim 1 , wherein the series of video images is captured by an endoscopic imaging system.
3 . The method of claim 1 , wherein the series of video images is captured by an open-field imaging system.
4 . The method of claim 1 , wherein the subset of surgical images includes an image depicting an event in the surgery.
5 . The method of claim 4 , wherein the subset of surgical images includes a single image depicting the event in the surgery.
6 . The method of claim 4 , wherein the event comprises: introduction of a surgical tool, removal of the surgical tool, movement of the surgical tool, identification of anatomical landmarks during surgery, critical view of safety in laparoscopic cholecystectomy, identification of critical structures during surgery, removal of organs, navigating through tissue structures as part of preparation, monitoring suture, checking for extravasation or leakage (blood, bile, or other fluids), cauterization, clipping, cutting, or any combination thereof.
7 . The method of claim 1 , further comprising: training a machine-learning model based on the subset of surgical images from the series of video images.
8 . The method of claim 7 , wherein the machine-learning model is a generative model, the method further comprising: generating one or more images using the trained machine-learning model.
9 . The method of claim 1 , further comprising: displaying the subset of surgical images from the series of video images.
10 . The method of claim 1 , further comprising:
detecting an event in an image in the subset of surgical images; and
storing a timestamp associated with the image.
11 . The method of claim 1 , wherein each hash value of the series of hash values is an N-bit binary representation.
12 . The method of claim 1 , wherein hashing image data for each image of the series of video images of the surgery comprises:
reducing the resolution of each image in the series of video images; and
after reducing the resolution, applying a hash algorithm to the image to obtain a corresponding hash value.
13 . The method of claim 12 , wherein the hash algorithm comprises: an average hash algorithm, a difference hash algorithm, a perceptual hash algorithm, a wavelet hash algorithm, a locality-sensitive hash algorithm, or any combination thereof.
14 . The method of claim 1 , wherein each difference value of the plurality of difference values is a Hamming distance.
15 . The method of claim 14 , wherein the Hamming distance between two hash values is computed by performing a bit-wise OR operation between the two hash values.
16 . The method of claim 1 , wherein clustering the plurality of difference values comprises performing probabilistic clustering, K-means clustering, fuzzy C-means clustering, mean-shift clustering, hierarchical clustering, or any combination thereof.
17 . The method of claim 16 , wherein performing probabilistic clustering comprises performing unsupervised learning of finite mixture models (FMMs).
18 . The method of claim 16 , wherein performing probabilistic clustering comprises:
(A) performing an expectation step to obtain an a posteriori probability for each cluster of a predefined number of clusters;
(B) performing a maximization step to obtain one or more parameters for each cluster of the predefined number of clusters; and
(C) repeating steps (A)-(B) until a convergence is reached.
19 . The method of claim 18 , wherein the one or more parameters comprises one or more distribution parameters.
20 . The method of claim 18 , wherein performing the maximization step further comprises calculating one or more prior probability values for each cluster of the predefined number of clusters.
21 . The method of claim 20 , wherein the one or more prior probability values include a spatial Markov Random Field (“MRF”) prior estimated from a posterior probability.
22 . The method of claim 1 , wherein selecting one or more image clusters from the plurality of image clusters comprises:
assigning each difference value of the plurality of difference values to one of the plurality of image clusters based on the maximum a posteriori (MAP) rule; and
ordering the plurality of image clusters.
23 . The method of claim 1 , wherein the first image of the series of video images is included in the subset of surgical images by default.
24 . The method of claim 1 , further comprising:
receiving a minimum frame selection window; and
including one or more images from an unselected image cluster to the subset of surgical images based on the minimum frame selection window.
25 . The method of claim 1 , further comprising: determining whether an image in the subset of surgical images comprises a motion artifact or noise.
26 . The method of claim 25 , further comprising: in accordance with a determination that the image comprises a motion artifact or noise, removing the image from the subset of surgical images.
27 . The method of claim 25 , further comprising: in accordance with a determination that the image comprises a motion artifact or noise, repairing the image.
28 . The method of claim 25 , further comprising: in accordance with a determination that the image comprises a motion artifact or noise, including the image in the subset of surgical images.
29 . A system for obtaining a subset of surgical images from a series of video images of a surgery, comprising:
one or more processors;
one or more memories; and
one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs including instructions for:
hashing image data for each image of the series of video images of the surgery to obtain a series of hash values;
calculating a plurality of difference values for the series of hash values, each of the plurality of difference values indicative of a difference between two consecutive hash values in the series of hash values;
generating a plurality of image clusters by clustering the plurality of difference values;
selecting one or more image clusters from the plurality of image clusters; and
producing the subset of surgical images from the series of video images using the selected one or more image clusters from the plurality of image clusters.
30 . A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions for obtaining a subset of surgical images from a series of video images of a surgery, which when executed by one or more processors of an electronic device, cause the electronic device to:
hash image data for each image of the series of video images of the surgery to obtain a series of hash values;
calculate a plurality of difference values for the series of hash values, each of the plurality of difference values indicative of a difference between two consecutive hash values in the series of hash values;
generate a plurality of image clusters by clustering the plurality of difference values;
select one or more image clusters from the plurality of image clusters; and
produce the subset of surgical images from the series of video images using the selected one or more image clusters from the plurality of image clusters.