Machine-learning for 3D object detection
A computer-implemented method of machine-learning for learning a neural network that encodes a super-point of a 3D point cloud into a latent vector. The method including obtaining a dataset of super-points. Each super-point is a set of points of a 3D point cloud. The set of points represents at least a part of an object. The method further includes learning the neural network based on the dataset of super-points. The learning includes minimizing a loss. The loss penalizes a disparity between two super-points. This constitutes improved machine-learning for 3D object detection.
1 . A computer-implemented method of applying a neural network learnt according to machine-learning for learning a neural network configured to encode a super-point of a 3D point cloud into a latent vector, the method of machine-learning including obtaining a dataset of super-points, each super-point being a set of points of a 3D point cloud, the set of points representing at least a part of an object, and learning the neural network based on the dataset of super-points, the learning comprising minimizing a loss penalizing a disparity between two super-points,
the method comprising:
obtaining one or more first super-points of a first 3D point cloud representing a 3D scene; and
obtaining one or more second super-points of a second 3D point cloud representing a 3D object;
encoding, by applying the neural network, the one or more first super-points each into a respective first latent vector and the one or more second super-points each into a respective second latent vector; and
determining a similarity between each first super-point of one or more first super-points and each second super-point of one or more second super-points, by, for said each first super-point and said each second super-point, computing a similarity measure between the respective first latent vector encoding the first super-point and the respective second latent vector encoding the second super-point,
wherein the obtaining the one or more first super-points further includes:
obtaining one or more initial super-points of the first 3D point cloud; and
filtering the one or more initial super-points by selecting, among the initial super-points, each initial super-point for which:
a disparity between dimensions of the respective initial super-point and dimensions of at least one second super-point of the one or more second super-points is smaller than a predefined threshold; and
a disparity between a position of the respective initial super-point and a position of at least one second super-point of the one or more second super-points is smaller than a predefined threshold,
the selected initial super-points being the one or more first super-points, and
wherein the filtering further includes selecting, among the initial super-points, each initial super-point for which:
a distance between each dimension of the super-point and a corresponding dimension of at least one second super-point of the one or more second super-points is smaller than a predefined threshold;
a ratio between each dimension of the super-point and a corresponding dimension of at least one second super-point of the one or more second super-points is smaller than a maximal ratio and larger than a minimal ratio; and
a difference between a relative height from a closest support surface of the super-point and a relative height from a closest support surface of at least one second super-point of the one or more second super-points is smaller than a predefined threshold.
2 . The method of use of claim 1 , wherein the method further comprises:
determining, among first super-points each having a determined similarity with at least one second super-point that is larger than a predefined threshold, one or more groups of first super-points, each group of first super-points having a similar shape than the second 3D point cloud.
3 . The method of claim 2 , wherein the determining of the one or more groups further comprises:
obtaining a graph of second super-points, the graph of second super-points having nodes each representing a second super-point and edges each representing a geometrical relation between the two super-points represented by the nodes connected by the edges and having one or more geometrical attributes of the geometrical relation; and
forming one or more graphs each having nodes representing each a first super-point, by constructing edges each connecting two nodes and each having one or more geometrical attributes similar to geometrical attributes of an edge of the graph of second super-points, each formed graph corresponding to a respective group.
4 . The method of claim 3 , further comprising, for each group, determining a similarity score by measuring a similarity of the first super-points in the group with the one or more second super-points.
5 . A device comprising:
a processor; and
a computer readable data storage medium having recorded thereon
a computer program comprising instructions
for performing machine-learning for learning a neural network configured for encoding a super-point of a 3D point cloud into a latent vector by the processor being configured to obtain a dataset of super-points, each super-point being a set of points of a 3D point cloud, the set of points representing at least a part of an object, and learn the neural network based on the dataset of super-points, the learning comprising minimizing a loss penalizing a disparity between two super-points, and/or
for applying a neural network learnable according to the machine-learning by the processor being configured to:
obtain one or more first super-points of a first 3D point cloud representing a 3D scene, and
obtain one or more second super-points of a second 3D point cloud representing a 3D object,
encode, by applying the neural network, the one or more first super-points each into a respective first latent vector and the one or more second super-points each into a respective second latent vector; and
determine a similarity between each first super-point of one or more first super-points and each second super-point of one or more second super-points, by, for said each first super-point and said each second super-point, computing a similarity measure between the respective first latent vector encoding the first super-point and the respective second latent vector encoding the second super-point,
wherein the processor being configured to obtain the one or more first super-points further by being further configured to:
obtain one or more initial super-points of the first 3D point cloud; and
filter the one or more initial super-points by selecting, among the initial super-points, each initial super-point for which:
a disparity between dimensions of the respective initial super-point and dimensions of at least one second super-point of the one or more second super-points is smaller than a predefined threshold; and
a disparity between a position of the respective initial super-point and a position of at least one second super-point of the one or more second super-points is smaller than a predefined threshold,
the selected initial super-points being the one or more first super-points, and
wherein the processor is further configured to filter by being configured to select, among the initial super-points, each initial super-point for which:
a distance between each dimension of the super-point and a corresponding dimension of at least one second super-point of the one or more second super-points is smaller than a predefined threshold;
a ratio between each dimension of the super-point and a corresponding dimension of at least one second super-point of the one or more second super-points is smaller than a maximal ratio and larger than a minimal ratio; and
a difference between a relative height from a closest support surface of the super-point and a relative height from a closest support surface of at least one second super-point of the one or more second super-points is smaller than a predefined threshold.
6 . The device of claim 5 , wherein in the machine-learning, the loss is a reconstruction loss and the loss penalizes a disparity between a super-point and a reconstruction of the super-point.
7 . The device of claim 6 , wherein the disparity is a distance between the super-point and the reconstruction of the super-point.
8 . The device of claim 5 , wherein the processor is further configured to determine, among first super-points each having a determined similarity with at least one second super-point that is larger than a predefined threshold, one or more groups of first super-points, each group of first super-points having a similar shape than the second 3D point cloud.
9 . The device of claim 8 , wherein the processor is further configured to determine of the one or more groups by being configured to:
obtain a graph of second super-points, the graph of second super-points having nodes each representing a second super-point and edges each representing a geometrical relation between the two super-points represented by the nodes connected by the edges and having one or more geometrical attributes of the geometrical relation, and
form one or more graphs each having nodes representing each a first super-point, by constructing edges each connecting two nodes and each having one or more geometrical attributes similar to geometrical attributes of an edge of the graph of second super-points, each formed graph corresponding to a respective group.
10 . The device of claim 9 , wherein, for each group, the processor further configured to determine a similarity score by measuring a similarity of the first super-points in the group with the one or more second super-points.
11 . A non-transitory computer readable medium having stored thereon a program that when executed by the computer causes the computer to implement the method according to claim 1 .
12 . The method of claim 1 , wherein in the method of machine-learning the loss is a reconstruction loss and the loss penalizes a disparity between a super-point and a reconstruction of the super-point.
13 . The method of claim 12 , wherein in the method of machine-learning the disparity is a distance between the super-point and the reconstruction of the super-point.
14 . The method of claim 13 , wherein in the method of machine-learning the distance is a Chamfer distance or an Earth-Mover distance.
15 . The method of claim 1 , wherein in the method of machine-learning the learning is an unsupervised learning.