Method and device for monitoring colonoscope withdrawal speed
A method for monitoring a colonoscope withdrawal speed, the method including: 1) acquiring, a real-time video of colonoscopy, decoding the video into images, cropping the images, and zooming-out the images cropped where texture information of the images is retained; 2) converting the images including the texture information to grayscale images; 3) obtaining Hash fingerprints of the images; 4) calculating a Hamming distance between the images; 5) comparing the Hash fingerprints of a current colonoscopy image with n previous colonoscopy images, to obtain an overlapping rate of the current colonoscopy image with any one of the n previous colonoscopy images; 6) calculating a weighted similarity of the images at a point in time t; 7) converting a weighted overlapping rate of the images at the point in time t into a stability coefficient; and 8) calculating a mean stability coefficient of colonoscopy images within a period of time 0-t.
1. A method, comprising:
1) acquiring, by endoscopic equipment, a real-time video of colonoscopy; decoding the video into images, cropping the images, and zooming-out the cropped images to remove the color and leave texture information of the images;
2) converting the images comprising the texture information to grayscale images;
3) obtaining Hash fingerprints of the grayscale images;
4) calculating a Hamming distance between the images;
5) comparing the Hash fingerprints of a current colonoscopy image with n previous colonoscopy images where n is a number, to obtain an overlapping rate of the current colonoscopy image with any one of the n previous colonoscopy images, thereby obtaining a similarity between the current colonoscopy image and the any one of the n previous colonoscopy images;
6) calculating a weighted similarity of the images at a point in time t, wherein the weighted similarity refers to a weighted average value of similarity between a current image and its previous 9 frames, thereby acquiring a representative value of the similarity within the point in time t, and the closer two frames are, the higher the weighted similarity is;
7) converting the weighted similarity of the images at the point in time t into a stability coefficient;
8) calculating a mean stability coefficient of colonoscopy images within a period of time 0-t as follows: the mean stability coefficient=100−weighted similarity, time 0 is a timepoint where the colonoscopy withdrawal process begins, and the time t is any given timepoint;
9) analyzing the real-time video of colonoscopy to obtain a first boundary between a standard colonoscopy video where a withdrawal time exceeds 6 min and a sub-standard colonoscopy video where a withdrawal time is between 5-6 min and a second boundary between the sub-standard colonoscopy video and a low-quality colonoscopy video where a withdrawal time is less than 5 min, respectively; and
10) monitoring in real time, according to the operations 1) to 8), a stability coefficient of a withdrawal process, and feeding back the stability coefficient to an operator, wherein: when the stability coefficient is less than the first boundary, indicating the withdrawal speed of the colonoscopy is within a normal range and the withdrawal speed meets the requirement of standard colonoscopy guideline; when the stability coefficient is greater than the first boundary, indicating the withdrawal speed of the colonoscopy is not within the normal range; when the stability coefficient is between the first boundary and the second boundary, a warning signal is presented in a dashboard monitoring real-time withdrawal speed during colonoscopy; and when the stability coefficient is greater than the second boundary, an emergency alarm is given.
2. The method of claim 1 , wherein in 1), the cropped images are zoomed-out by bicubic interpolation.
3. The method of claim 1 , wherein in 2), the images are converted to the grayscale images by the following equation: Gray=0.30×R+0.59×G+0.11×B; where R, G and B represent brightness values of red color, green color and blue color, respectively.
4. The method of claim 1 , wherein in 3), the Hash fingerprints of the images are obtained by difference Hashing (dHash).
5. The method of claim 1 , wherein in 5), the overlapping rate of the current colonoscopy image with any one of the n previous colonoscopy images is calculated by the following equation:
Sim
=
1
0
0
×
(
6
4
-
d
(
x
,
y
)
)
6
4
,
where d(x, y) represents the Hamming distance between different images, d(x, y)=Σx⊕y, x and y represent character strings corresponding to different images, respectively, referring to the Hash fingerprints of the images, and ⊕ represents exclusive OR.
6. The method of claim 1 , wherein in 7), the weighted overlapping rate at the point in time t is converted into the stability coefficient by the following equation: ESim=100− Sim , where Sim represents the weighted similarity of the images at the point in time t.
7. A device, comprising:
an image acquisition module, configured to acquire, by endoscopic equipment, a real-time video of colonoscopy, decode the video into images, crop the images, and zoom-out the cropped images wherein texture information of the images is retained;
a grayscale image conversion module, configured to convert the images comprising the texture information to grayscale images;
a Hash fingerprint acquisition module, configured to obtain Hash fingerprints of the images;
a Hamming distance calculation module, configured to calculate a Hamming distance between different images;
a similarity calculation module, configured to compare the Hash fingerprints of a current colonoscopy image with n previous colonoscopy images, to obtain an overlapping rate of the current colonoscopy image with any one of the n previous colonoscopy images, thereby obtaining a similarity between the current colonoscopy image and the any one of the n previous colonoscopy images;
a weighted similarity calculation module, configured to calculate a weighted similarity of the images at a point in time t;
a stability coefficient conversion module, configured to convert the weighted overlapping rate at the point in time t into a stability coefficient;
a mean stability coefficient conversion module, configured to calculate a mean stability coefficient of colonoscopy images within a period of time 0-t;
a colonoscopy video analysis module, configured to analyze the real-time video of colonoscopy to obtain a first boundary between a standard colonoscopy video and a sub-standard colonoscopy video and a second boundary between the sub-standard colonoscopy video and a low-quality colonoscopy video, respectively; and
a withdrawal speed feedback module, configured to monitor in real time a stability coefficient of a withdrawal by a physician performing the colonoscopy, and feed the stability coefficient to the physician, wherein: when the stability coefficient is less than the first boundary, indicating the withdrawal speed of the colonoscopy is within a normal range; when the stability coefficient is greater than the first boundary, indicating the withdrawal speed of the colonoscopy is not within the normal range; when the stability coefficient is between the first boundary and the second boundary, a warning signal is given; and when the stability coefficient is greater than the second boundary, an emergency alarm is given.
8. The device of claim 7 , wherein in the similarity calculation module, the overlapping rate of the current colonoscopy image with any one of the n previous colonoscopy images is calculated by the following equation:
Sim
=
1
0
0
×
(
6
4
-
d
(
x
,
y
)
)
6
4
,
where d(x, y) represents the Hamming distance between different images, d(x, y)=Σx⊕y, x and y represent character strings corresponding to different images, respectively, referring to the Hash fingerprints of the images, and ⊕ represents exclusive OR.
9. The device of claim 7 , wherein in the stability coefficient conversion module, the weighted overlapping rate at the point in time t is converted into the stability coefficient by the following equation: ESim=100− Sim , where Sim represents the weighted similarity of the images at the point in time t.