Image processing device, image processing method and storage medium storing program for tracking an object
An image-processing device performing: extracting features from a target image based on a shape, the shape including sub parts each being a group of feature points each being correlated with a position within an object; deriving an estimated shape by applying a global model to an initial shape, the global model representing a relation between one of the features extracted and a motion of the initial shape; detecting, in the sub parts, an occluded sub part by comparing an estimated shape derived from a previous image and an estimated shape derived from a current image; and generating a final output shape by combining estimated shapes each derived by applying local models to the initial shape, the local models each representing a relation between the feature and a motion of a sub part in the sub parts and each restricting a motion of at least one of the sub parts.
1. An image processing device comprising:
a memory that stores a set of instructions; and
at least one processor configured to execute the set of instructions to:
extract features of an object from a target image comprising the object based on a shape representing the object, the shape including sub parts, each sub part being a group of feature points, each feature point being correlated with a position within the object;
derive an estimated shape for the object by applying a global model to an initial shape within the target image, the global model representing a relation between one of the features extracted and a motion of the initial shape within the target image;
detect, in the sub parts, an occluded sub part by comparing the estimated shape for the object to an estimated shape for the object derived from an image previous to the target image;
select a local model to be applied to the occluded sub part that restricts the motion of the occluded sub part, wherein the local model represents a relation between the features and a motion of the occluded sub part of the sub parts and restricts a motion of at least one of the sub parts; and
generate a final output shape by combining an estimated shape that is derived by applying the selected local model to the occluded sub part and the estimated shape derived for the object by applying the global model.
2. The image processing device according to claim 1 , wherein
the at least one processor is configured to:
set the initial shape as a first start shape, and derive the estimated shape by repetition of calculating a motion of a start shape by applying the global model to the features extracted from the image based on the start shape, and setting, as a next start shape, an incremental shape where the motion calculated is added to the start shape.
3. The image processing device according to claim 1 wherein the at least one processor is configured to:
learn the global and the local models by one or more series of training images and shapes which are given as true shapes.
4. The image processing device according to claim 1 , wherein the initial shape is an estimated shape derived from a previous image of the target image.
5. The image processing device according to claim 1 , wherein the shape and the sub parts are represented by position information of the feature points.
6. An image processing method comprising:
extracting features of an object from a target image comprising the object based on a shape representing the object, the shape including sub parts, each sub part being a group of feature points, each feature point being correlated with a position within the object;
deriving an estimated shape for the object by applying a global model to an initial shape within the target image, the global model representing a relation between one of the features extracted and a motion of the initial shape within the target image;
detecting, in the sub parts, an occluded sub part by comparing the estimated shape for the object to an estimated shape for the object derived from an image previous to the target image;
selecting a local model to be applied to the occluded sub part that restricts the motion of the occluded sub part, wherein the local model represents a relation between the features and a motion of the occluded sub part of the sub parts and restricts a motion of at least one of the sub parts, and
generate a final output shape by combining an estimated shape that is derived by applying the selected local model to the occluded sub part and the estimated shape derived for the object by applying the global model.
7. A non-transitory computer-readable medium storing a program that causes a computer to execute:
feature extraction processing of extracting features of an object from a target image comprising the object based on a shape representing the object, the shape including sub parts, each sub part being a group of feature points, each feature point being correlated with a position within the object;
alignment processing of deriving an estimated shape for the object by applying a global model to an initial shape within the target image, the global model representing a relation between one of the features extracted and a motion of the initial shape within the target image;
occlusion detection processing of detecting, in the sub parts, an occluded sub part by comparing the estimated shape for the object and to an estimated shape for the object derived from an image previous to the target image;
estimation processing of selecting a local model to be applied to the occluded sub part that restricts the motion of the occluded sub part, wherein the local model represents a relation between the features and a motion of the occluded sub part of the sub parts and restricts a motion of at least one of the sub parts; and
generate a final output shape by combining an estimated shape that is derived by applying the selected local model to the occluded sub part and the estimated shape derived for the object by applying the global model.
8. The non-transitory computer-readable medium according to claim 7 , wherein
the alignment processing sets the initial shape as a first start shape, and derive the estimated shape by repetition of calculating a motion of a start shape by applying the global model to the features extracted from the image based on the start shape, and setting, as a next start shape, an incremental shape where the motion calculated is added to the start shape.
9. The non-transitory computer-readable medium according to claim 7 , storing a program that causes a computer to operate as:
learning processing of learning the global and the local models by one or more series of training images and shapes which are given as true shapes.
10. The non-transitory computer-readable medium according to claim 7 , wherein
the initial shape is an estimated shape derived from a previous image of the target image.