Systems and methods for processing real-time video from a medical image device and detecting objects in the video
The present disclosure relates to systems and methods for processing real-time video and detecting objects in the video. In one implementation, a system is provided that includes an input port for receiving real-time video obtained from a medical image device, a first bus for transferring the received real-time video, and at least one processor configured to receive the real-time video from the first bus, perform object detection by applying a trained neural network on frames of the received real-time video, and overlay a border indicating a location of at least one detected object in the frames. The system also includes a second bus for receiving the video with the overlaid border, an output port for outputting the video with the overlaid border from the second bus to an external display, and a third bus for directly transmitting the received real-time video to the output port.
1. A computer-implemented system for processing real-time video, the system comprising:
an input port for receiving the real-time video obtained from a medical image device, the real-time video comprising a plurality of frames;
a first bus for transferring the received real-time video;
at least one processor configured to perform operations consisting of the following for object detection and border overlay:
receive the real-time video from the first bus,
feed the plurality of frames of the received real-time video directly to a trained neural network, perform object detection by applying the trained neural network to the plurality of frames of the received real-time video, and overlay a border indicating a location of at least one detected object in the plurality of frames determined by applying the trained neural network to the plurality of frames of the received real-time video, the overlaid border comprising a graphical indicator with a first pattern and/or color displayed around a region of the plurality of frames including the at least one detected object; and
modify the graphical indicator to be displayed in one or more other patterns and/or colors based on whether the at least one detected object is a true positive or a false positive;
a second bus for receiving the plurality of frames of the real-time video with the overlaid border; and
an output port for outputting the plurality of frames of the real-time video with the overlaid border from the second bus to an external display.
2. The system of claim 1 , further comprising a third bus for directly transmitting the received real-time video to the output port.
3. The system of claim 2 , wherein the third bus is activated upon receipt of an error signal from the at least one processor.
4. The system of claim 1 , wherein the at least one detected object is an abnormality.
5. The system of claim 4 , wherein the abnormality comprises a change in human tissue from one type of cell to another type of cell.
6. The system of claim 4 , wherein the abnormality comprises an absence of human tissue from a location where the human tissue is expected.
7. The system of claim 4 , wherein the abnormality comprises a formation on or of human tissue.
8. The system of claim 7 , wherein the abnormality comprises a lesion.
9. The system of claim 8 , wherein the lesion comprises a polypoid lesion.
10. The system of claim 8 , wherein the lesion comprises a non-polypoid lesion.
11. The system of claim 1 , wherein the at least one processor is further configured to modify the graphical indicator to be displayed in a second pattern and/or color when the at least one detected object is a true positive and further modify the graphical indicator to be displayed in a third pattern and/or color if the at least one detected object is a false positive.
12. The system of claim 1 , wherein the overlaid border comprises a graphical pattern around a region of the plurality of frames including the at least one detected object, the graphical pattern being displayed in a first color.
13. The system of claim 12 , wherein the at least one processor is further configured to, after an elapsed period of time, modify the graphical pattern to be displayed in a second color when the at least one detected object is a true positive and further modify the graphical pattern to be displayed in a third color if the at least one detected object is a false positive.
14. The system of claim 13 , wherein the at least one processor is further configured to transmit a command to one or more speakers to produce a sound when the graphical pattern of the overlaid border is modified.
15. The system of claim 14 , wherein at least one of duration, tone, frequency, and amplitude of the sound depends on whether the at least one detected object is a true positive or a false positive.
16. The system of claim 13 , wherein the at least one processor is further configured to transmit a command to at least one wearable apparatus to vibrate when the graphical pattern of the overlaid border is modified.
17. The system of claim 16 , wherein at least one of duration, frequency, and amplitude of the vibration depends on whether the at least one detected object is a true positive or a false positive.