Systems and methods for characterizing object pose detection and measurement systems

1. A method for characterizing a pose estimation system, comprising:

receiving, from a pose estimation system configured to estimate poses of objects with respect to a reference coordinate system, by a characterization system comprising a processor and a memory, a first plurality of poses of an arrangement of objects in a first scene;

receiving, from the pose estimation system, by the characterization system, a second plurality of poses of the arrangement of objects in a second scene, the second scene being a rigid transformation of the arrangement of objects of the first scene with respect to the pose estimation system;

computing, by the characterization system, a coarse scene transformation between the first scene and the second scene;

matching, by the characterization system, corresponding poses between the first plurality of poses and the second plurality of poses;

computing, by the characterization system, a refined scene transformation between the first scene and the second scene based on coarse scene transformation, the first poses, and the second poses;

transforming, by the characterization system, the first plurality of poses, received from the pose estimation system, based on the refined scene transformation to compute a plurality of transformed first poses; and

computing an average rotation error and an average translation error of the pose estimation system based on differences between the transformed first poses and the second plurality of poses received from the pose estimation system.

2. The method of claim 1 , wherein the rigid transformation of the arrangement of objects with respect to the pose estimation system comprises: a rotation of the arrangement of objects.

3. The method of claim 1 , wherein the arrangement of objects is on a support platform, and

wherein the characterization system is configured to control the support platform to rigidly transform the arrangement of objects with respect to the pose estimation system.

4. The method of claim 1 , wherein a fiducial, adjacent the arrangement of objects, is imaged in the first scene, rigidly transformed with the arrangement of objects, and imaged in the second scene, and

wherein the coarse scene transformation between the first scene and the second scene is computed based on computing a first pose of the fiducial imaged in the first scene and a second pose of the fiducial imaged in the second scene.

5. The method of claim 1 , wherein the matching the corresponding poses between the first plurality of poses and the second plurality of poses is performed by:

transforming the first plurality of poses in accordance with the coarse scene transformation to compute a plurality of coarsely transformed first poses; and

for each coarsely transformed first pose of the first coarsely transformed first poses:

identifying a second pose of the second poses closest to the coarsely transformed first pose; and

determining that the transformed first pose and the second pose closest to the coarsely transformed first pose match when a distance between the coarsely transformed first pose and the second pose closest to the coarsely transformed first pose is less than a false-positive threshold distance.

6. The method of claim 1 , wherein the matching the corresponding poses between the first plurality of poses and the second plurality of poses is performed by:

transforming the first plurality of poses in accordance with the coarse scene transformation to compute a plurality of coarsely transformed first poses; and

for each coarsely transformed first pose of the first coarsely transformed first poses:

identifying a second pose of the second poses closest to the coarsely transformed first pose;

identifying a type of an object corresponding to the coarsely transformed first pose and the second pose;

positioning a first 3-D model of the type of the object at the coarsely transformed first pose;

positioning a second 3-D model of the type of the object at the second pose; and

determining that the coarsely transformed first pose and the second pose closest to the coarsely transformed first pose match when an intersection between the positioned first 3-D model and the positioned second 3-D model satisfies a false-positive threshold intersection.

7. The method of claim 1 , wherein the computing the refined scene transformation comprises:

initializing a current scene transformation based on the coarse scene transformation;

computing a plurality of first poses as transformed by the current scene transformation; and

updating the current scene transformation in accordance with reducing a cost function computed based on differences between the second poses and the first poses as transformed by the current scene transformation.

8. The method of claim 1 , wherein the average rotation error is computed based on a sum of the rotation errors between the differences between rotational components of the transformed first poses and the second plurality of poses, and

wherein the average translation error is computed based on a sum of the translation errors between the differences between translation components of the transformed first poses and the second plurality of poses.

9. The method of claim 8 , wherein the average rotation error R err is computed in accordance with:

R

e

⁢

r

⁢

r

=

1

n

⁢

∑

i



R

⁡

(

(

P

S

1

i

⁢

T

S

1

⁢

S

2

)

-

1

⁢

Q

S

2

i

)



,

and

wherein the average translation error T err is computed in accordance with:

T

e

⁢

r

⁢

r

=

1

n

⁢

∑

i



T

⁡

(

(

P

S

1

i

⁢

T

S

1

⁢

S

2

)

-

1

⁢

Q

S

2

i

)



,

where P S 1 i is an i-th pose of the first poses, Q S 2 i is an i-th pose of the second poses, T S 1 S 2 is the refined scene transformation from the first scene S 1 to the second scene S 2 , and n is the number of first poses and the number of second poses.

10. A system for characterizing a pose estimation system, comprising:

a processor; and

a memory storing instructions that, when executed by the processor, cause the processor to:

receive, from a pose estimation system configured to estimate poses of objects with respect to a reference coordinate system, a first plurality of poses of an arrangement of objects in a first scene;

receive, from the pose estimation system, a second plurality of poses of the arrangement of objects in a second scene, the second scene being a rigid transformation of the arrangement of objects of the first scene with respect to the pose estimation system;

compute a coarse scene transformation between the first scene and the second scene;

match corresponding poses between the first plurality of poses and the second plurality of poses;

compute a refined scene transformation between the first scene and the second scene based on coarse scene transformation, the first poses, and the second poses;

transform the first plurality of poses, received from the pose estimation system, based on the refined scene transformation to compute a plurality of transformed first poses; and

compute an average rotation error and an average translation error of the pose estimation system based on differences between the transformed first poses and the second plurality of poses received from the pose estimation system.

11. The system of claim 10 , wherein the rigid transformation of the arrangement of objects with respect to the pose estimation system comprises a rotation of the arrangement of objects.

12. The system of claim 10 , further comprising a support platform, and

wherein the memory further stores instructions that, when executed by the processor, cause the processor to control the support platform to rigidly transform the arrangement of objects with respect to the pose estimation system from the first scene to the second scene.

13. The system of claim 10 , wherein a fiducial, adjacent the arrangement of objects, is imaged in the first scene, rigidly transformed with the arrangement of objects, and imaged in the second scene, and

wherein the coarse scene transformation between the first scene and the second scene is computed based on computing a first pose of the fiducial imaged in the first scene and a second pose of the fiducial imaged in the second scene.

14. The system of claim 10 , wherein the memory further stores instructions that, when executed by the processor, cause the processor to match the corresponding poses between the first plurality of poses and the second plurality of poses by:

transforming the first plurality of poses in accordance with the coarse scene transformation to compute a plurality of transformed first poses; and

for each transformed first pose of the first transformed first poses:

identifying a second pose of the second poses closest to the transformed first pose; and

determining that the transformed first pose and the second pose closest to the transformed first pose match when a distance between the transformed first pose and the second pose closest to the transformed first pose is less than a false-positive threshold distance.

15. The system of claim 10 , wherein the memory further stores instructions that, when executed by the processor, cause the processor to match the corresponding poses between the first plurality of poses and the second plurality of poses by:

transforming the first plurality of poses in accordance with the coarse scene transformation to compute a plurality of transformed first poses; and

for each transformed first pose of the first transformed first poses:

identifying a second pose of the second poses closest to the transformed first pose;

identifying a type of an object corresponding to the transformed first pose and the second pose;

positioning a first 3-D model of the type of the object at the transformed first pose;

positioning a second 3-D model of the type of the object at the second pose; and

determining that the transformed first pose and the second pose closest to the transformed first pose match when an intersection between the positioned first 3-D model and the positioned second 3-D model satisfies a false-positive threshold intersection.

16. The system of claim 10 , wherein the memory further stores instructions that, when executed by the processor, cause the processor to compute the refined scene transformation by:

initializing a current scene transformation based on the coarse scene transformation;

computing a plurality of first poses as transformed by the current scene transformation; and

updating the current scene transformation in accordance with reducing a cost function computed based on differences between the second poses and the first poses as transformed by the current scene transformation.

17. The system of claim 10 , wherein the memory further stores instructions that, when executed by the processor, cause the processor to:

compute the average rotation error based on a sum of the rotation errors between the differences between rotational components of the transformed first poses and the second plurality of poses, and

compute the average translation error based on a sum of the translation errors between the differences between translation components of the transformed first poses and the second plurality of poses.

18. The system of claim 17 , wherein the average rotation error R err is computed in accordance with:

R

e

⁢

r

⁢

r

=

1

n

⁢

∑

i



R

⁡

(

(

P

S

1

i

⁢

T

S

1

⁢

S

2

)

-

1

⁢

Q

S

2

i

)



,

and

wherein the average translation error T err is computed in accordance with:

T

e

⁢

r

⁢

r

=

1

n

⁢

∑

i



T

⁡

(

(

P

S

1

i

⁢

T

S

1

⁢

S

2

)

-

1

⁢

Q

S

2

i

)



,

where P S 1 i is an i-th pose of the first poses, Q S 2 i is an i-th pose of the second poses, T S 1 S 2 is the refined scene transformation from the first scene S 1 to the second scene S 2 , and n is the number of first poses and the number of second poses.