System and method for reviewing capsule images with detected regions of interest
Systems and methods are provided for processing images captured by a capsule camera system. According to one method, a frame sequence comprises panoramic picture representative of a GI scene covering substantially a lateral 360-degree Field-of-View is received. CADe (computer-assisted detection) is applied to detect one or more target abnormalities in the frame sequence, where said applying the computer-assisted detection (CADe) results in one or more CADe frames and each CADe frame contains at least one detection area containing a detected abnormality or a false positive. Circle shift is applied laterally to a target CADe frame to shift a detection area to a lateral center or close to the lateral center of the target CADe frame. The CADe frames comprising the target CADe frame are provided with the detection area shifted to the lateral center or close to the lateral center of the target CADe frame.
1. A method for processing a frame sequence corresponding to gastrointestinal (GI) images, the method comprising:
receiving the frame sequence, wherein each frame of the frame sequence comprises a panoramic picture representative of a GI scene covering substantially a lateral 360-degree field of view (FoV);
applying CADe (computer-assisted detection) to detect one or more target abnormalities in the frame sequence, wherein said applying CADe to detect said one or more target abnormalities in the frame sequence generates one or more CADe frames and each CADe frame contains at least one detection area comprising a detected abnormality or a false positive, and wherein one or more frames in the frame sequence are skipped for at least one pair of consecutive CADe frames;
determining horizontal location of said at least one detection area in a target CADe frame;
in response to the horizontal location of said at least one detection area in the target CADe frame, applying circle shift laterally to the target CADe frame to shift said at least one detection area to a lateral center or close to the lateral center of the target CADe frame; and
providing the CADe frames comprising the target CADe frame with said at least one detection area shifted to the lateral center or close to the lateral center of the target CADe frame; and
wherein said applying CADe to detect said one or more target abnormalities in the frame sequence, said determining horizontal location of said at least one detection area and said applying circle shift laterally to the target CADe frame are performed using program codes executable on one or more electronic devices or processors.
2. The method of claim 1 , further comprising displaying the CADe frames on a display device.
3. The method of claim 2 , wherein the detection area is highlighted.
4. The method of claim 3 , wherein the detection area is highlighted using a bounding box.
5. The method of claim 2 , wherein when the target CADe frame comprises two or more detection areas, the detection area closest to the lateral center of the CADe frames is highlighted and remaining detection areas are not highlighted.
6. The method of claim 5 , wherein at least one additional CADe frame is generated for one of the remaining detection areas and displayed, and wherein said one of the remaining detection areas is highlighted and circularly shifted laterally to near a center of said at least one additional CADe frame and other detection areas are not highlighted.
7. The method of claim 2 , wherein for at least two or more closely located detection areas, at least two of said two or more closely located detection areas are highlighted jointly.
8. The method of claim 7 , wherein said at least two or more detection areas are highlighted by using a single bounding box to encompass said at least two or more detection areas.
9. The method of claim 2 , wherein when one real abnormality appears at least in part in two detection areas at two edges of one target CADe frame prior to the circle shift, the two detection areas are united into one single detection area after the circle shift and said single detection area is highlighted.
10. The method of claim 2 , wherein the CADe frames are displayed as a CADe sequence separated from the frame sequence and when one CADe frame displayed in a display window on a display device is selected for further viewing, the display window is switched to display the frame sequence or another display window is used to display the frame sequence.
11. The method of claim 10 , wherein after the display window is switched to display the frame sequence or said another display window is used to display the frame sequence, a starting frame of the frame sequence corresponding to the target CADe frame is displayed.
12. The method of claim 11 , wherein the starting frame is initially displayed with lateral shifting by a same distance as the target CADe frame.
13. The method of claim 12 , wherein the starting frame is displayed without the lateral shifting after a period.
14. The method of claim 10 , wherein the detection area is not highlighted after the display window is switched to display the frame sequence.
15. The method of claim 2 , wherein when the frame sequence is displayed as a video, intermediate frames between a first target CADe frame and a next target CADe frame are displayed at a slower frame rate if a detection area distance between the first target CADe frame and the next target CADe frame is larger.
16. The method of claim 2 , wherein when the frame sequence is displayed as a video and at least one CADe frame with one detection area is displayed for a period t and nearby non-CADe frames in the frame sequence are display for shorter periods.
17. The method of claim 16 , wherein when the target CADe frame contains more detection areas, the target CADe frame is displayed for a longer viewing time.
18. The method of claim 2 , wherein when the frame sequence is displayed, multiple display windows or a larger display window is used to display multiple frames of the frame sequence simultaneously to reduce viewing time.
19. The method of claim 2 , wherein the frame sequence is displayed as a video and the frame sequence comprises intermediate frames without detection areas (non-CADe) between a first target CADe frame shifted by a first shift amount and a second target CADe frame shifted by a second shift amount, the intermediate frames without detection areas are displayed with laterally and circularly shifting in increments corresponding to fractions of a difference between the first shift amount and the second shift amount to avoid large shift between any two frames from the first target CADe frame to the next target CADe frame.
20. The method of claim 19 , wherein the increments associated with said laterally and circularly shifting are substantially equal.
21. The method of claim 19 , wherein the increments associated with said laterally and circularly shifting increase and then decrease between the first target CADe frame and the second target CADe frame.
22. A system for processing a frame sequence corresponding to gastrointestinal (GI) images, the system comprising:
an interface module coupled to receive the frame sequence, wherein each of the frame sequence comprises a panoramic picture representative of a GI scene covering substantially a lateral 360-degree field of view (FoV);
one or more processing modules coupled to the interface module, wherein said one or more processing modules are configured to:
apply CADe (computer-assisted detection) to detect one or more target abnormalities in the frame sequence, wherein CADe applied to detect said one or more target abnormalities in the frame sequence generates one or more CADe frames and each CADe frame contains at least one detection area comprising a detected abnormality or a false positive, and wherein one or more frames in the frame sequence are skipped for at least one pair of consecutive CADe frames;
determine horizontal location of said at least one detection area in a target CADe frame;
in response to the horizontal location of said at least one detection area in the target CADe frame, apply circle shift laterally to a target CADe frame to shift a detection area to a lateral center or close to the lateral center of the target CADe frame; and
provide the CADe frames comprising the target CADe frame with said at least one detection area shifted to the lateral center or close to the lateral center of the target CADe frame; and
wherein said applying CADe to detect said one or more target abnormalities in the frame sequence, said determining horizontal location of said at least one detection area and said applying circle shift laterally to the target CADe frame are performed using program codes executable on one or more electronic devices or processors.
23. The system of claim 22 , further comprising a display device coupled to said one or more processing modules to display the CADe frames comprising the target CADe frame with the detection area shifted to the lateral center or close to the lateral center of the target CADe frame.