Tracking device, endoscope system, and tracking method

1. A tracking device comprising a processor including hardware,

the processor being configured to:

set a start frame to start tracking of a tracking target in a video including multiple frames;

extract multiple representative points of a contour of the tracking target in the start frame;

track the extracted multiple representative points in frames subsequent to the start frame;

perform outlier determination on the tracked multiple representative points based on an interrelationship of the multiple representative points;

perform a process of removing an outlier representative point that is a representative point determined to be an outlier;

update the representative points by extracting new representative points based on multiple representative points after the process of removing the outlier representative point, in a case where any frame subsequent to the start frame meets a given condition; and

determines a deviation degree of a first representative point of the multiple representative points based on any one of relative distance information between the first representative point and one or more of the representative points adjacent in a direction along the contour, and a curvature of a curve formed by the first representative point and some of the representative points adjacent in the direction along the contour, so as to determine whether the first representative point is the outlier representative point.

2. The tracking device as defined in claim 1 , wherein

the processor extracts multiple representative points of a contour of a tag region tagged as the tracking target to the start frame.

3. The tracking device as defined in claim 1 , wherein

the processor extracts the multiple representative points from the contour of the tracking target such that adjacent representative points are spaced apart at a given interval.

4. The tracking device as defined in claim 1 , wherein

the processor extracts the multiple representative points such that, on the contour of the tracking target, a density of representative points at a portion with a large curvature of the contour is higher than a density of representative points at a portion with a small curvature of the contour.

5. The tracking device as defined in claim 1 , wherein

the processor extracts the new representative points based on the multiple representative points after the process of removing the outlier representative point, when a number of representative points becomes equal to or smaller than a given number threshold value due to the process of removing the outlier representative point and generates a closed curve based on the multiple representative points after the process of removing the outlier representative point, and extracts the new representative points from the generated closed curve.

6. The tracking device as defined in claim 1 , wherein

the processor extracts the new representative points based on the multiple representative points after the process of removing the outlier representative point, when reliability of a tracking result becomes equal to or lower than a given reliability threshold value.

7. The tracking device as defined in claim 1 , wherein

the processor extracts the new representative points based on the multiple representative points after the process of removing the outlier representative point at a given time interval.

8. The tracking device as defined in claim 1 , wherein

the processor generates annotation data in which an inside of a closed curve generated based on the tracked multiple representative points is defined as an annotation region for each frame subsequent to the start frame.

9. An endoscope system comprising:

a memory that stores a trained model;

an endoscopic scope that captures a detection image; and

a processor that accepts the detection image as input, and performs a process of detecting a position of a given object from the detection image by using the trained model,

the trained model having been trained by machine learning based on training data in which annotation data is associated with a frame image in a video,

the annotation data being generated by:

acquiring the video including multiple frames;

setting a start frame to start tracking of a tracking target;

extracting multiple representative points of a contour of the tracking target in the start frame;

tracking the extracted multiple representative points in frames subsequent to the start frame;

performing outlier determination on the tracked multiple representative points based on an interrelationship of the multiple representative points;

performing a process of removing an outlier representative point that is a representative point determined to be an outlier;

updating the representative points by extracting new representative points based on multiple representative points after the process of removing the outlier representative point, in a case where any frame subsequent to the start frame meets a given condition;

determining a deviation degree of a first representative point of the multiple representative points based on any one of relative distance information between the first representative point and one or more of the representative points adjacent in a direction along the contour, and a curvature of a curve formed by the first representative point and some of the representative points adjacent in the direction along the contour, so as to determine whether the first representative point is the outlier representative point; and

generating the annotation data in which an inside of a closed curve generated based on the tracked multiple representative points is defined as an annotation region for each frame subsequent to the start frame.

10. A tracking method comprising:

acquiring a video including multiple frames;

setting a start frame to start tracking of a tracking target;

extracting multiple representative points of a contour of the tracking target in the start frame;

tracking the extracted multiple representative points in frames subsequent to the start frame;

performing outlier determination on the tracked multiple representative points based on an interrelationship of the multiple representative points;

performing a process of removing an outlier representative point that is a representative point determined to be an outlier;

updating the representative points by extracting new representative points based on multiple representative points after the process of removing the outlier representative point, in a case where any frame subsequent to the start frame meets a given condition; and

determining a deviation degree of a first representative point of the multiple representative points based on any one of relative distance information between the first representative point and one or more of the representative points adjacent in a direction along the contour, and a curvature of a curve formed by the first representative point and some of the representative points adjacent in the direction along the contour, so as to determine whether the first representative point is the outlier representative point.