Computer program, object tracking method, and object tracking device

1. A non-transitory computer readable medium that embodies instructions causing one or more processors to perform steps comprising:

acquiring, from a camera, a plurality of image frames of an object and a scene, the scene being a background of the object;

deriving a first 3D pose of the object based on: (i) at least one image in the plurality of image frames, and (ii) at least one 2D template based on a 3D model corresponding to the object without the scene;

deriving 3D scene feature points of the scene based at least on a first image and a second image in the plurality of image frames, the first image being captured with the camera positioned at a first position and the second image being captured with the camera positioned at a second position after the first image is captured;

deriving a second 3D pose of the object based on: (a) the second image, and (b) a most recent 3D pose of the object;

obtaining a 3D-2D relationship between 3D points represented in a 3D coordinate system and image feature points on the second image; and

updating the derived second pose based on the 3D-2D relationship, wherein

the 3D points include the 3D scene feature points and 3D model points on the 3D model.

2. The non-transitory computer readable medium according to claim 1 , wherein deriving the 3D scene feature points is performed based on a triangulation method or a bundle adjustment method.

3. The non-transitory computer readable medium according to claim 1 , wherein the method further comprises:

detecting a 3D pose of the object in the first image.

4. A method for tracking an object, comprising:

acquiring, from a camera, a plurality of image frames of an object and a scene, the scene being a background of the object;

deriving, by a processor, a first 3D pose of the object based on: (i) at least one image in the plurality of image frames, and (ii) at least one 2D template based on a 3D model corresponding to the object without the scene;

deriving, by the processor, 3D scene feature points of the scene based at least on a first image and a second image in the plurality of image frames, the first image being captured with the camera positioned at a first position and the second image being captured with the camera positioned at a second position after the first image is captured;

deriving, by the processor, a second 3D pose of the object based on: (a) the second image, and (b) a most recent 3D pose of the object;

obtaining, by the processor, a 3D-2D relationship between 3D points represented in a 3D coordinate system and image feature points on the second image; and

updating, by the processor, the derived second pose based on the 3D-2D relationship, wherein

the 3D points include the 3D scene feature points and 3D model points on the 3D model.

5. An object tracking device comprising:

a camera configured to acquire a plurality of image frames of an object and a scene, the scene being a background of the object; and

a processor programmed to:

derive a first 3D pose of the object based on: (i) at least one image in the plurality of image frames, and (ii) at least one 2D template based on a 3D model corresponding to the object without the scene;

derive 3D scene feature points of the scene based at least on the first image and the second image in the plurality of image frames, the first image being captured with the camera positioned at a first position and the second image being captured with the camera positioned at a second position after the first image is captured;

derive a second 3D pose of the object based on: (a) the second image, and (b) a most recent 3D pose of the object;

obtain a 3D-2D relationship between 3D points represented in a 3D coordinate system and image feature points on the second image; and

update the derived second pose based on the 3D-2D relationship, wherein

the 3D points include the 3D scene feature points and 3D model points on the 3D model.