Systems and methods for pose detection and measurement
A method for estimating a pose of an object includes: receiving a plurality of images of the object captured from multiple viewpoints with respect to the object; initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images; predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images; and computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints associated with a 3-D model of the object as arranged in accordance with the current pose, and as projected to each of the viewpoints.
1. A method for estimating a pose of an object comprising:
receiving a plurality of images of the object captured from multiple viewpoints with respect to the object;
initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images;
predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images; and
computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints associated with a 3-D model of the object as arranged in accordance with the current pose, and as projected to each of the viewpoints,
wherein each of the plurality of differences corresponds to a different viewpoint of the viewpoints, and
wherein each of the differences is computed for a viewpoint of the viewpoints based on a difference between:
the plurality of 2-D keypoints associated with an image of the plurality of images corresponding to the viewpoint; and
projecting the 3-D keypoints of the 3-D model arranged in the current pose to the viewpoint,
wherein the cost function is:
min
R
o
,
T
o
∑
i
:
N
,
j
:
M
c
i
j
[
u
ij
,
v
ij
]
-
K
j
[
R
j
T
j
]
[
R
o
T
o
]
[
x
i
,
y
i
,
z
i
]
wherein R o , T o is the pose of the object, i is an index iterating through N 3-D keypoints, j is an index iterating through M viewpoints, c ij is a confidence for a corresponding keypoint prediction [u ij , v ij ] of an i-th keypoint in an image for corresponding to a j-th viewpoint, R j , T j , K j are extrinsic parameters and intrinsic parameters of a j-th camera used to capture an image from a j-th viewpoint, and [x i , y i , z i ] is an i-th 3-D keypoint.
2. A method for estimating a pose of an object comprising:
receiving a plurality of images of the object captured from multiple viewpoints with respect to the object;
initializing a current pose of the object based on computing an initial estimated pose of the object from at least one of the plurality of images;
predicting a plurality of 2-D keypoints associated with the object from each of the plurality of images; and
computing an updated pose that minimizes a cost function based on a plurality of differences between the 2-D keypoints and a plurality of 3-D keypoints associated with a 3-D model of the object as arranged in accordance with the current pose, and as projected to each of the viewpoints,
wherein each of the plurality of differences corresponds to a different viewpoint of the viewpoints, and
wherein each of the differences is computed for a viewpoint of the viewpoints based on a difference between:
the plurality of 2-D keypoints associated with an image of the plurality of images corresponding to the viewpoint; and
projecting the 3-D keypoints of the 3-D model arranged in the current pose to the viewpoint,
wherein the cost function further accounts for symmetries in the object in accordance with:
min
R
o
,
T
o
∑
i
:
N
,
j
:
M
min
S
~
v
j
c
i
j
[
u
ij
,
v
ij
]
-
K
j
S
[
R
j
T
j
]
[
R
o
T
o
]
[
x
i
,
y
i
,
z
i
]
wherein R o , T o is the pose of the object, i is an index iterating through N 3-D keypoints, j is an index iterating through M viewpoints, c ij is a confidence for a corresponding keypoint prediction [u ij , v ij ] of an i-th keypoint in an image from a j-th viewpoint, R j , T j , K j are extrinsic parameters and intrinsic parameters of the j-th camera used to capture the images from a j-th viewpoint, [x i , y i , z i ] is an i-th 3-D keypoint, S is a transform between different symmetries of the object, and v j is the j-th view.