Estimator training method and pose estimating method using depth image
An estimator training method and a pose estimating method using a depth image are disclosed, in which the estimator training method may train an estimator configured to estimate a pose of an object, based on an association between synthetic data and real data, and the pose estimating method may estimate the pose of the object using the trained estimator.
1. A method of estimating a pose, the method comprising:
obtaining, using at least one processor, a plurality of vector sets corresponding to a plurality of patches included in an input image using an estimator trained based on an association between synthetic data in which an object is synthesized and real data in which the object is photographed; and
estimating, using the at least one processor, a pose of an input object included in the input image based on the plurality of vector sets by generating a plurality of mixture models corresponding to a plurality of joints included in the input object based on the plurality of vector sets.
2. The method of claim 1 , wherein each of the plurality of vector sets comprises a plurality of vectors respectively indicating the plurality of joints included in the input object.
3. The method of claim 1 , wherein the estimating comprises:
generating a plurality of 2-part Gaussian mixture models (GMMs) corresponding to a plurality of joints included in the input object based on the plurality of vector sets; and
calculating three-dimensional (3D) coordinates of the plurality of joints included in the input object based on the plurality of 2-part GMMs.
4. The method of claim 3 , wherein the calculating comprises:
comparing an average value of a first Gaussian component to an average value of a second Gaussian component included in each of the plurality of 2-part GMMs;
detecting a 2-part GMM having a difference between the average value of the first Gaussian component and the average value of the second Gaussian component less than a threshold value, among the plurality of 2-part GMMs; and
calculating 3D coordinates of a joint corresponding to the detected 2-part GMM based on an average value of a Gaussian component having a greater weight between a first Gaussian component and a second Gaussian component included in the detected 2-part GMM.
5. The method of claim 3 , wherein the calculating comprises:
comparing an average value of a first Gaussian component to an average value of a second Gaussian component included in each of the plurality of 2-part GMMs;
detecting first GMMs, each having a difference between the average value of the first Gaussian component and the average value of the second Gaussian component less than a threshold value, among the plurality of 2-part GMMs;
detecting a second GMM having the difference between the average value of the first Gaussian component and the average value of the second Gaussian component greater than or equal to the threshold value;
detecting a Gaussian component closest to the first GMMs among N Gaussian components included in an N-part GMM corresponding to a view of the second GMM, N being an integer greater than “2”;
selecting one of a first Gaussian component and a second Gaussian component included in the second GMM based on the closest Gaussian component; and
calculating 3D coordinates of a joint corresponding to the second GMM based on the closest Gaussian component and the selected Gaussian component.
6. The method of claim 5 , wherein the calculating of the 3D coordinates of the plurality of joints further comprises selecting the N-part GMM from a plurality of N-part GMMs corresponding to a plurality of views based on the view of the second GMM,
wherein the plurality of N-part GMMs is generated in advance using a dataset comprising information related to the plurality of joints included in the object.
7. An apparatus for estimating a pose, the apparatus comprising:
at least one processor configured to:
obtain a plurality of vector sets corresponding to a plurality of patches included in an input image using an estimator trained based on an association between synthetic data in which an object is synthesized and real data in which the object is photographed; and
estimate a pose of an input object included in the input image based on the plurality of vector sets, by generating a plurality of mixture models corresponding to a plurality of joints included in the input object based on the plurality of vector sets.