Tracking system and tracking method for water-surface objects, and marine vessel including tracking system for water-surface objects
A tracking system for tracking water-surface objects includes a stereo camera on a hull, at least one memory, and at least one processor coupled to the at least one memory. The at least one processor is configured or programmed to detect at least one object based on a first image and a second image captured by a first imaging unit and a second imaging unit of the stereo camera, and set one detected object as a tracking target in a third image captured by the first imaging unit, the second imaging unit or another imaging unit. The at least one processor is further configured or programmed to track the tracking target using a temporal change in a feature of the tracking target, and use at least one object detected based on the first image and the second image during tracking to correct the tracking result.
1. A tracking system for tracking water-surface objects, the tracking system comprising:
a stereo camera including a first imaging unit and a second imaging unit, the stereo camera being disposed on a hull;
at least one memory; and
at least one processor coupled to the at least one memory and configured or programmed to:
acquire a first image captured by either one of the first imaging unit or the second imaging unit;
acquire a second image captured by the other one of the first imaging unit or the second imaging unit;
acquire a third image captured by any one of the first imaging unit, the second imaging unit, or another imaging unit disposed on the hull;
detect at least one object based on the first image and the second image;
set one of the detected at least one object as a tracking target in the third image;
track the tracking target in the third image by using a temporal change in a feature of the tracking target in the third image; and
use at least one object detected based on the first image and the second image in a process of tracking the tracking target in the third image to correct a tracking result of the tracking target in the third image.
2. The tracking system according to claim 1 , wherein the at least one processor is further configured or programmed to:
acquire positional information of the tracking target based on the tracking result, the positional information including a relative distance of the tracking target from the stereo camera and an azimuth of the tracking target with respect to the stereo camera; and
output the acquired positional information.
3. The tracking system according to claim 2 , wherein the at least one processor is further configured or programmed to:
in acquiring the positional information, acquire the azimuth of the tracking target based on a position in a left-and-right direction of the tracking target in the third image, and acquire the relative distance based on an image portion corresponding to the tracking target in the first image and an image portion corresponding to the tracking target in the second image.
4. The tracking system according to claim 2 , further comprising a navigation controller configured or programmed to control navigation of the hull by using the output positional information.
5. The tracking system according to claim 1 , wherein the at least one processor is further configured or programmed to, in correcting the tracking result of the tracking target:
set a first frame in the third image indicating a position of the tracking target as the tracking result, and set a second frame in the third image indicating a position of one of the at least one object detected based on the first image and the second image; and
correct the tracking result by updating a position and a shape of the first frame based on the second frame in the process of tracking the tracking target.
6. The tracking system according to claim 5 , wherein the at least one processor is further configured or programmed to, in correcting the tracking result of the tracking target:
set a third frame in the third image based on the second frame; and
correct the tracking result by updating a position and a shape of the third frame based on the second frame and updating the position and the shape of the first frame based on the third frame in the process of tracking the tracking target.
7. The tracking system according to claim 6 , wherein the at least one processor is further configured or programmed to, in correcting the tracking result of the tracking target:
every time when acquiring one of a sequence of the third images, update the third frame so as to reduce a difference between the third frame and the second frame by a predetermined ratio, and every time when acquiring one of a sequence of the third images, update the first frame to match the third frame in a case in which a difference between the first frame and the third frame exceeds a threshold value.
8. The tracking system according to claim 5 , further comprising a display to display the third image; wherein
the at least one processor is further configured or programmed to:
in correcting the tracking result of the tracking target, cause the display to display the third image on which the first frame and the second frame are superimposed.
9. The tracking system according to claim 1 , wherein the at least one processor is further configured or programmed to:
track the tracking target by using a Kernelized Correlation Filter algorithm.
10. A tracking method for tracking water-surface objects for use in a marine vessel, the marine vessel including a stereo camera including a first imaging unit and a second imaging unit, the stereo camera being disposed on a hull, the tracking method comprising:
acquiring a first image captured by either one of the first imaging unit or the second imaging unit;
acquiring a second image captured by the other one of the first imaging unit or the second imaging unit;
acquiring a third image captured by any one of the first imaging unit, the second imaging unit, or another imaging unit disposed on the hull;
detecting at least one object based on the first image and the second image;
setting one of the detected at least one object as a tracking target in the third image;
tracking the tracking target in the third image by using a temporal change in a feature of the tracking target in the third image; and
using at least one object detected based on the first image and the second image in a process of tracking the tracking target in the third image to correct a tracking result of the tracking target in the third image.
11. A marine vessel comprising:
a hull; and
a tracking system for tracking water-surface objects, the tracking system including:
a stereo camera including a first imaging unit and a second imaging unit, the stereo camera being disposed on the hull;
at least one memory; and
at least one processor coupled to the at least one memory and configured or programmed to:
acquire a first image captured by either one of the first imaging unit or the second imaging unit;
acquire a second image captured by the other one of the first imaging unit or the second imaging unit;
acquire a third image captured by any one of the first imaging unit, the second imaging unit, or another imaging unit disposed on the hull;
detect at least one object based on the first image and the second image;
set one of the detected at least one object as a tracking target in the third image;
track the tracking target in the third image by using a temporal change in a feature of the tracking target in the third image; and
use at least one object detected based on the first image and the second image in a process of tracking the tracking target in the third image to correct a tracking result of the tracking target in the third image.
12. The tracking system according to claim 1 , wherein the third image is captured by the another imaging unit disposed on the hull.