Methods and apparatus for determining the pose of an object based on point cloud data
Methods, apparatus, and computer readable media that are related to 3D object detection and pose determination and that may optionally increase the robustness and/or efficiency of the 3D object recognition and pose determination. Some implementations are generally directed to techniques for generating an object model of an object based on model point cloud data of the object. Some implementations of the present disclosure are additionally and/or alternatively directed to techniques for application of acquired 3D scene point cloud data to a stored object model of an object to detect the object and/or determine the pose of the object.
1. A method implemented by one or more processors, the method comprising:
obtaining a three-dimensional point cloud of an object;
selecting, from the three-dimensional point cloud, a model point pair that includes a model reference point and an additional model point;
generating a feature vector for the model point pair based on the model point pair, wherein the feature vector defines a plurality of features of the model point pair and defines each of the plurality of features at a corresponding discretization level;
storing, in one or more non-transitory computer readable media, a mapping of the feature vector to the model point pair;
determining an additional feature vector that is the same as the feature vector, but that includes a given value, for a given feature of the features, that varies from the feature vector by a single level of the corresponding discretization level of the given feature;
storing, in the one or more non-transitory computer readable media, a mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector; and
controlling a robot using the mapping.
2. The method of claim 1 , wherein the mapping of the feature vector to the model point pair is a mapping of a hash value to the model point pair.
3. The method of claim 2 , wherein the hash value is generated based on the feature vector.
4. The method of claim 1 , wherein the mapping of the feature vector to the additional feature vector is a mapping of a first hash key to a second hash key,
wherein the first hash key is generated based on the feature vector, and
wherein the second hash key is generated based on the additional feature vector.
5. The method of claim 1 , wherein controlling the robot using the mapping comprises:
receiving scene point cloud data that captures at least a portion of the object in an environment;
determining a pose of the object in the environment based on the scene point cloud data and the mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector; and
adjusting the robot based on the determined pose of the object in the environment.
6. The method of claim 5 , wherein determining the pose of the object based on the scene point cloud data and the mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector comprises:
determining a scene point pair from the scene point cloud data;
determining a feature vector of the scene point pair maps to the feature vector;
selecting the one or more additional point pairs based on the mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector;
generating one or more candidate in-plane rotations based on the scene point pair and the one or more additional point pairs; and
determining the pose of the object based on the one or more candidate in-plane rotations and reference points of the corresponding point pairs.
7. A system, comprising:
memory storing instructions;
one or more processors executing the instructions to:
obtain a three-dimensional point cloud of an object;
select, from the three-dimensional point cloud, a model point pair that includes a model reference point and an additional model point;
generate a feature vector for the model point pair based on the model point pair, wherein the feature vector defines a plurality of features of the model point pair and defines each of the plurality of features at a corresponding discretization level;
store, in one or more non-transitory computer readable media, a mapping of the feature vector to the model point pair;
determine an additional feature vector that is the same as the feature vector, but that includes a given value, for a given feature of the features, that varies from the feature vector by a single level of the corresponding discretization level of the given feature; and
store, in the one or more non-transitory computer readable media, a mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector; and
control a robot using the mapping.
8. The system of claim 7 , wherein the mapping of the feature vector to the model point pair is a mapping of a hash value to the model point pair.
9. The system of claim 8 , wherein the hash value is generated based on the feature vector.
10. The system of claim 7 , wherein the mapping of the feature vector to the additional feature vector is a mapping of a first hash key to a second hash key,
wherein the first hash key is generated based on the feature vector, and
wherein the second hash key is generated based on the additional feature vector.
11. The system of claim 7 , wherein one or more of the processors, in executing the instructions to use the mapping in controlling a robot, are to:
receive scene point cloud data that captures at least a portion of the object in an environment;
determine a pose of the object in the environment based on the scene point cloud data and the mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector; and
adjust the robot based on the determined pose of the object in the environment.
12. The system of claim 11 , wherein in determining the pose of the object based on the scene point cloud data and the mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector, one or more of the processors are to:
determine a scene point pair from the scene point cloud data;
determine a feature vector of the scene point pair maps to the feature vector;
select the one or more additional point pairs based on the mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector;
generate one or more candidate in-plane rotations based on the scene point pair and the one or more additional point pairs; and
determine the pose of the object based on the one or more candidate in-plane rotations and reference points of the corresponding point pairs.
13. One or more non-transitory computer-readable media comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:
obtaining a three-dimensional point cloud of an object;
selecting, from the three-dimensional point cloud, a model point pair that includes a model reference point and an additional model point;
generating a feature vector for the model point pair based on the model point pair, wherein the feature vector defines a plurality of features of the model point pair and defines each of the plurality of features at a corresponding discretization level;
storing, in one or more non-transitory computer readable media, a mapping of the feature vector to the model point pair;
determining an additional feature vector that is the same as the feature vector, but that includes a given value, for a given feature of the features, that varies from the feature vector by a single level of the corresponding discretization level of the given feature;
storing, in the one or more non-transitory computer readable media, a mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector; and
controlling a robot using the mapping.
14. The non-transitory computer-readable media of claim 13 , wherein the mapping of the feature vector to the model point pair is a mapping of a hash value to the model point pair.
15. The non-transitory computer-readable media of claim 14 , wherein the hash value is generated based on the feature vector.
16. The non-transitory computer-readable media of claim 13 , wherein the mapping of the feature vector to the additional feature vector is a mapping of a first hash key to a second hash key,
wherein the first hash key is generated based on the feature vector, and
wherein the second hash key is generated based on the additional feature vector.
17. The non-transitory computer-readable media of claim 13 , wherein execution of the instructions in controlling the robot using the mapping cause the one or more processors to perform the following operations:
receiving scene point cloud data that captures at least a portion of the object in an environment;
determining a pose of the object in the environment based on the scene point cloud data and the mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector; and
adjusting the robot based on the determined pose of the object in the environment.
18. The non-transitory computer-readable media of claim 17 , wherein determining the pose of the object based on the scene point cloud data and the mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector comprises:
determining a scene point pair from the scene point cloud data;
determining a feature vector of the scene point pair maps to the feature vector;
selecting the one or more additional point pairs based on the mapping of the feature vector to the additional feature vector or to one or more additional point pairs mapped to by the additional feature vector;
generating one or more candidate in-plane rotations based on the scene point pair and the one or more additional point pairs; and
determining the pose of the object based on the one or more candidate in-plane rotations and reference points of the corresponding point pairs.