Combined compression and feature extraction models for storing and analyzing medical videos
A method of compressing and detecting target features of a medical video is presented herein. In some embodiments, the method may include receiving an uncompressed medical video comprising at least one target feature, compressing the uncompressed medical video to generate a compressed medical video based on a predicted location of the at least one target feature using a first pretrained machine learning model, and detecting the location of the at least one target feature of the compressed medical video using a second pretrained machine learning model. In some embodiments, the first pretrained machine learning model and the second pretrained machine learning model may be trained in tandem using domain-specific medical videos.
1 . A method comprising:
receiving an uncompressed medical video comprising at least one target feature;
compressing the uncompressed medical video to generate a compressed medical video based on a predicted location of the at least one target feature using a first pretrained machine learning model; and,
detecting a location of the at least one target feature of the compressed medical video using a second pretrained machine learning model,
wherein the first pretrained machine learning model and the second pretrained machine learning model are trained in tandem using domain-specific medical videos in a manner that, in a first iteration, a training medical video is both compressed and analyzed to detect a location of at least one training target feature before a start of a second iteration after the first iteration, and
wherein the first pretrained machine learning model is trained to compress a first video frame differently from a second video frame, the first video frame comprising the location of at least one training target feature and the second video frame not comprising the location of at least one training target feature.
2 . The method of claim 1 , wherein the uncompressed medical video is obtained during a colonoscopy.
3 . The method of claim 2 , wherein one or more target features of the at least one target feature is a polyp.
4 . The method of claim 1 , wherein the first pretrained machine learning model comprises a video compression transformer.
5 . The method of claim 1 , wherein the second pretrained machine learning model comprises a RetinaNet detector.
6 . The method of claim 1 , wherein the first pretrained machine learning model is configured to allocate a larger proportion of bits to the predicted location of the at least one target feature than to other parts of the compressed medical video.
7 . The method of claim 1 , wherein the second pretrained machine learning model is designed to optimize an accuracy of detection of the at least one target feature.
8 . The method of claim 1 , wherein the first pretrained machine learning model is designed to optimize an accuracy of detection of the at least one target feature and to minimize a compression loss of the compressed medical video.
9 . A system comprising:
a memory configured to store a plurality of processor-executable instructions, the memory including:
a video compressor based on a first pretrained machine learning model;
a target feature detector based on a second pretrained machine learning model; and,
a processor configured to execute the plurality of processor-executable instructions instruction to perform operations including:
receiving an uncompressed medical video comprising at least one target feature;
compressing the uncompressed medical video to generate a compressed medical video based on a predicted location of the at least one target feature; and,
detecting a location of the at least one target feature of the compressed medical video,
wherein the first pretrained machine learning model and the second pretrained machine learning model are trained in tandem using domain-specific medical videos in a manner that, in a first iteration, a training medical video is both compressed and analyzed to detect a location of at least one training target feature before a start of a second iteration after the first iteration, and
wherein the first pretrained machine learning model is trained to compress a first video frame differently from a second video frame, the first video frame comprising the location of at least one training target feature and the second video frame not comprising the location of at least one training target feature.
10 . The system of claim 9 , wherein the first pretrained machine learning model comprises a video compression transformer.
11 . The system of claim 9 , wherein the second pretrained machine learning model comprises a RetinaNet detector.
12 . The system of claim 9 , wherein the second pretrained machine learning model is designed to optimize an accuracy of detection of the at least one target feature.
13 . The system of claim 9 , wherein the first pretrained machine learning model is designed to optimize an accuracy of detection of the at least one target feature and to minimize a compression loss of the compressed medical video.
14 . A non-transitory processor-readable storage medium storing a plurality of processor-executable instructions, the plurality of processor-executable instructions being executed by a processor to perform operations comprising:
receiving an uncompressed medical video comprising at least one target feature;
compressing the uncompressed medical video to generate a compressed medical video based on a predicted location of the at least one target feature using a first pretrained machine learning model; and,
detecting a location of the at least one target feature of the compressed medical video using a second pretrained machine learning model,
wherein the first pretrained machine learning model and the second pretrained machine learning model are trained in tandem using domain-specific medical videos in a manner that, in a first iteration, a training medical video is both compressed and analyzed to detect a location of at least one training target feature before a start of a second iteration after the first iteration, and
wherein the first pretrained machine learning model is trained to compress a first video frame differently from a second video frame, the first video frame comprising the location of at least one training target feature and the second video frame not comprising the location of at least one training target feature.
15 . The non-transitory processor-readable storage medium of claim 14 , wherein the first pretrained machine learning model comprises a video compression transformer.
16 . The non-transitory processor-readable storage medium of claim 14 , wherein the second pretrained machine learning model comprises a RetinaNet detector.
17 . The non-transitory processor-readable storage medium of claim 14 , wherein the first pretrained machine learning model is designed to optimize an accuracy of detection of the at least one target feature and to minimize a compression loss of the compressed medical video and the second pretrained machine learning model is designed to optimize a second accuracy of detection of the at least one target feature.