Tracking device, endoscope system, and tracking method

1. A tracking device comprising:

a processor including hardware,

the processor being configured to:

set a start frame and an end frame in a video including multiple frames;

perform forward tracking of tracking a tracking target in a forward direction in frames subsequent to the start frame based on a region of the tracking target in the start frame;

perform backward tracking of tracking the tracking target in a backward direction in frames previous to the end frame based on a region of the tracking target in the end frame; and

generate a combined mask image indicating a region of the tracking target in each of the frames in which the forward tracking and the backward tracking have been performed by combining a forward mask image based on the forward tracking with a backward mask image based on the backward tracking,

wherein the processor performs a display process of superimposing the combined mask image on the video such that a non-overlapping portion where the forward mask image and the backward mask image do not overlap each other in the combined mask image has a higher transparency than an overlapping portion where the forward mask image and the backward mask image overlap each other in the combined mask image.

2. The tracking device as defined in claim 1 , wherein

the region of the tracking target in the start frame and the region of the tracking target in the end frame are set based on tagging data added to the tracking target.

3. The tracking device as defined in claim 1 , wherein

the processor

sets weight information that causes the forward mask image to be weighted more than the backward mask image in a frame closer to the start frame,

sets the weight information that causes the backward mask image to be weighted more than the forward mask image in a frame closer to the end frame, and

generates the combined mask image by combining the forward mask image with the backward mask image based on the weight information.

4. The tracking device as defined in claim 1 , wherein

the processor

applies the forward mask image as the combined mask image in a predetermined number of frames subsequent to the start frame, and

applies the backward mask image as the combined mask image in a predetermined number of frames previous to the end frame.

5. The tracking device as defined in claim 1 , wherein

the processor

extracts start tracking points from a contour of the tracking target in the start frame and performs the forward tracking based on the extracted start tracking points, and

extracts end tracking points from a contour of the tracking target in the end frame and performs the backward tracking based on the extracted end tracking points.

6. The tracking device as defined in claim 5 , wherein

the forward mask image is an image identifying a region inside a closed curve that connects a plurality of points as a result of tracking the start tracking points, and

the backward mask image is an image identifying a region inside a closed curve that connects a plurality of points as a result of tracking the end tracking points.

7. The tracking device as defined in claim 1 , wherein

the processor combines the forward mask image with the backward mask image based on a weighting coefficient that is dependent on a frame position in the video and a pixel position in an image.

8. The tracking device as defined in claim 7 , wherein

the processor generates the combined mask image using a forward weighting coefficient wf and a backward weighting coefficient wb and based on the following expression:

I ( t,x )= wf ( t,x )× If ( t,x )+ wb ( t,x )× Ib ( t,x )

where x is the pixel position, t is the frame position, I is the combined mask image, If is the forward mask image, and Ib is the backward mask image.

9. A tracking method comprising:

setting a start frame and an end frame in a video including multiple frames;

performing forward tracking of tracking a tracking target in a forward direction in frames subsequent to the start frame based on a region of the tracking target in the start frame;

performing backward tracking of tracking the tracking target in a backward direction in frames previous to the end frame based on a region of the tracking target in the end frame;

generating a combined mask image indicating a region of the tracking target in each of the frames in which the forward tracking and the backward tracking have been performed by combining a forward mask image based on the forward tracking with a backward mask image based on the backward tracking; and

performing display process of superimposing the combined mask image on the video such that a non-overlapping portion where the forward mask image and the backward mask image do not overlap each other in the combined mask image has a higher transparency than an overlapping portion where the forward mask image and the backward mask image overlap each other in the combined mask image.

10. The tracking method as defined in claim 9 , wherein

the region of the tracking target in the start frame and the region of the tracking target in the end frame are set based on tagging data added to the tracking target.

11. The tracking method as defined in claim 9 , further comprising:

setting weight information that causes the forward mask image to be weighted more than the backward mask image in a frame closer to the start frame,

setting the weight information that causes the backward mask image to be weighted more than the forward mask image in a frame closer to the end frame, and

generating the combined mask image by combining the forward mask image with the backward mask image based on the weight information.

12. The tracking method as defined in claim 9 , further comprising:

applying the forward mask image as the combined mask image in a predetermined number of frames subsequent to the start frame, and

applying the backward mask image as the combined mask image in a predetermined number of frames previous to the end frame.

13. The tracking method as defined in claim 9 , further comprising:

extracting start tracking points from a contour of the tracking target in the start frame and performing the forward tracking based on the extracted start tracking points, and

extracting end tracking points from a contour of the tracking target in the end frame and performing the backward tracking based on the extracted end tracking points.

14. The tracking method as defined in claim 13 , wherein

the forward mask image is an image identifying a region inside a closed curve that connects a plurality of points as a result of tracking the start tracking points, and

the backward mask image is an image identifying a region inside a closed curve that connects a plurality of points as a result of tracking the end tracking points.

15. The tracking method as defined in claim 9 , further comprising:

combining the forward mask image with the backward mask image based on a weighting coefficient that is dependent on a frame position in the video and a pixel position in an image.

16. The tracking method as defined in claim 15 , further comprising:

generating the combined mask image using a forward weighting coefficient wf and a backward weighting coefficient wb and based on the following expression:

I ( t,x )= wf ( t,x )× If ( t,x )+ wb ( t,x )× Ib ( t,x )

where x is the pixel position, t is the frame position, I is the combined mask image, If is the forward mask image, and Ib is the backward mask image.

17. A tracking method comprising:

setting a start frame and an end frame in a video including multiple frames;

performing forward tracking of tracking a tracking target in a forward direction in frames subsequent to the start frame based on a region of the tracking target in the start frame;

performing backward tracking of tracking the tracking target in a backward direction in frames previous to the end frame based on a region of the tracking target in the end frame; and

generating a combined mask image indicating a region of the tracking target in each of the frames in which the forward tracking and the backward tracking have been performed by combining a forward mask image based on the forward tracking with a backward mask image based on the backward tracking, and by combining the forward mask image with the backward mask image based on a weighting coefficient that is dependent on a frame position in the video and a pixel position in an image,

wherein the generating step comprises generating the combined mask image using a forward weighting coefficient wf and a backward weighting coefficient wb and based on the following expression:

I ( t,x )= wf ( t,x )× If ( t,x )+ wb ( t,x )× Ib ( t,x )

where x is the pixel position, t is the frame position, I is the combined mask image, If is the forward mask image, and Ib is the backward mask image.

18. The tracking method as defined in claim 17 , wherein

the region of the tracking target in the start frame and the region of the tracking target in the end frame are set based on tagging data added to the tracking target.

19. The tracking method as defined in claim 17 , further comprising:

setting weight information that causes the forward mask image to be weighted more than the backward mask image in a frame closer to the start frame,

setting the weight information that causes the backward mask image to be weighted more than the forward mask image in a frame closer to the end frame, and

generating the combined mask image by combining the forward mask image with the backward mask image based on the weight information.

20. The tracking method as defined in claim 17 , further comprising:

applying the forward mask image as the combined mask image in a predetermined number of frames subsequent to the start frame, and

applying the backward mask image as the combined mask image in a predetermined number of frames previous to the end frame.