Machine-learning for 3D segmentation
A computer-implemented method of machine-learning including obtaining a dataset of training samples. Each training sample includes a pair of 3D modeled object portions labelled with a respective value. The respective value indicates whether or not the two portions belong to a same segment of a 3D modeled object. The method further includes learning a neural network based on the dataset. The neural network takes as input two portions of a 3D modeled object representing a mechanical part and outputs a respective value. The respective value indicates an extent to which the two portions belong to a same segment of the 3D modeled object. The neural network is thereby usable for 3D segmentation. The method constitutes an improved solution for 3D segmentation.
1. A computer-implemented method of machine-learning, the method comprising:
obtaining a dataset of training samples each including a pair of 3D modeled object portions labelled with a respective value that indicates whether or not two portions belong to a same segment of a 3D modeled object; and
learning a neural network based on the dataset, the neural network being configured to
take as input two portions of a 3D modeled object representing a mechanical part, and
output a respective value that indicates an extent to which the two portions belong to a same segment of the 3D modeled object, the neural network being thereby usable for 3D segmentation,
wherein the learning of the neural network includes minimizing a loss comprising a term penalizing, for pairs of the dataset, a disparity between a respective value that indicates whether or not the two portions of a pair belong to a same segment of a 3D modeled object and a second respective value output by the neural network for the two portions,
wherein the neural network transforms the input two portions each into a corresponding intermediary feature, and the loss further includes another term penalizing, for said pairs of the dataset, a disparity between the respective value that indicates whether or not the two portions of a pair belong to a same segment of a 3D modeled object and a proximity between the intermediary features corresponding to the two portions, and
wherein the term is of a type:
∑
i
=
1
N
d
CE
(
g
w
(
f
w
(
X
i
(
1
)
)
,
f
w
(
X
i
(
2
)
)
)
,
p
i
)
,
where ((X 1 (1) , X 2 (2) ), . . . (X N (1) , X N (2) )) are pairs of the dataset, g w (f w (X i (1) ), f w (X i (2) )) is the respective value outputted by the neural network for the pair (X i (1) , X i (2) ), g w ∘f w is the neural network, p i is the respective value that indicates whether or not portions X i (1) and X i (2) belong to a same segment of a 3D modeled object, and d CE is a loss penalizing a disparity between g w (f w (X i (1) ), f w (X i (2) )) and p i ;
and the other term is of the type:
∑
i
=
1
N
d
M
(
f
w
(
X
i
(
1
)
)
,
f
w
(
X
i
(
2
)
)
,
p
i
)
,
where, f w (X i (1) ) and f w (X i (2) ) are the intermediary features corresponding to portions X i (1) and X i (2) of the pair (X i (1) , X i (2) ), and d M is a loss penalizing a disparity between p i and a proximity between f w (X i (1) ) and f w (X i (2) ).
2. The method of claim 1 , wherein the neural network includes:
an encoding neural network configured to take as input a portion of a 3D modeled object and to encode the portion into a latent vector, and
a predictive neural network configured to take as input a pair of latent vectors outputted by the encoding neural network and to output a respective value that indicates an extent to which the two portions encoded by the latent vectors belong to a same segment of the 3D modeled object.
3. The method of claim 2 , wherein the encoding neural network is configured to take as input a portion of a 3D modeled object and neighboring portions of the input portion.
4. The method of claim 1 , wherein the obtaining of the dataset includes:
obtaining an initial dataset of segmented 3D modeled objects; and
for each segmented 3D modeled object of the initial dataset, generating pairs of portions of the segmented 3D modeled object, the portions of a pair belonging either to a same segment of the segmented 3D modeled object or to different segments, each pair including a respective value that indicates whether or not the two portions belong to a same segment of a 3D modeled object.
5. The method of claim 4 , wherein for one or more segmented 3D modeled objects of the initial dataset, the generating of the pairs includes generating pairs of portions for which at least one portion does not belong entirely to a single segment.
6. The method of claim 4 , wherein the obtaining of the dataset further includes creating mini-batches of pairs, the learning being carried out mini-batch by mini-batch.
7. A computer-implemented method of machine-learning for learning a neural network, the method comprising:
obtaining a dataset of training samples each including a pair of 3D modeled object portions labelled with a respective value that indicates whether or not two portions belong to a same segment of a 3D modeled object; and
learning the neural network based on the dataset, the neural network being configured to take as input two portions of a 3D modeled object representing a mechanical part and to output a respective value that indicates an extent to which the two portions belong to a same segment of the 3D modeled object, the neural network being thereby usable for 3D segmentation,
wherein the 3D segmentation comprises:
obtaining a 3D modeled object representing the mechanical part,
applying the neural network to pairs of portions of the 3D modeled object, thereby outputting a set of values, each for a respective pair of portions of the 3D modeled object, each indicating an extent to which the two portions belong to a same segment of the 3D modeled object, and
performing a segmentation of the 3D modeled object based on a set of one or more similarity indicators between neighboring portions of the 3D modeled object, the set of similarity indicators including the set of values,
wherein the learning of the neural network includes minimizing a loss comprising a term penalizing, for pairs of the dataset, a disparity between a respective value that indicates whether or not the two portions of a pair belong to a same segment of a 3D modeled object and a second respective value output by the neural network for the two portions.
wherein the neural network transforms the input two portions each into a corresponding intermediary feature, and the loss further includes another term penalizing, for said pairs of the dataset, a disparity between the respective value that indicates whether or not the two portions of a pair belong to a same segment of a 3D modeled object and a proximity between the intermediary features corresponding to the two portions, and
wherein the term is of a type:
∑
i
=
1
N
d
CE
(
g
w
(
f
w
(
X
i
(
1
)
)
,
f
w
(
X
i
(
2
)
)
)
,
p
i
)
,
where ((X 1 (1) , X 2 (2) ), . . . (X N (1) , X N (2) )) are pairs of the dataset, g w (f w (X i (1) ), f w (X i (2) )) is the respective value outputted by the neural network for the pair (X i (1) , X i (2) ), g w ∘f w is the neural network, p i is the respective value that indicates whether or not portions X i (1) and X i (2) belong to a same segment of a 3D modeled object, and d CE is a loss penalizing a disparity between g w (f w (X i (1) ), f w (X i (2) )) and p i ,
and the other term is of the type:
∑
i
=
1
N
d
M
(
f
w
(
X
i
(
1
)
)
,
f
w
(
X
i
(
2
)
)
,
p
i
)
,
where, f w (X i (1) ) and f w (X i (2) ) are the intermediary features corresponding to portions X i (1) and X i (2) of the pair (X i (1) , X i (2) ), and d M is a loss penalizing a disparity between p i and a proximity between f w (X i (1) ) and f w (X i (2) ).
8. The method of use of claim 7 , wherein the performing of the segmentation further comprises:
performing a network-based segmentation of the 3D modeled object based on the set of values and then another segmentation of the 3D modeled object based on at least one other similarity indicator, or performing said another segmentation and then the network-based segmentation, or
performing the segmentation of the 3D modeled object based only on the set of values, or
performing the segmentation of the 3D modeled object based on the set of values combined with at least one other similarity indicator.
9. A device comprising:
a processor; and
a computer-readable data storage medium having recorded thereon a computer program comprising instructions for machine-learning that when executed by the processor causes the processor to be configured to:
obtain a dataset of training samples each including a pair of 3D modeled object portions labelled with a respective value that indicates whether or not two portions belong to a same segment of a 3D modeled object, and
learn a neural network based on the dataset, the neural network being configured to take as input two portions of a 3D modeled object representing a mechanical part and to output a respective value that indicates an extent to which the two portions belong to a same segment of the 3D modeled object, the neural network being thereby usable for 3D segmentation; and/or
implement 3D segmentation by the processor being configured to:
obtain a 3D modeled object representing a mechanical part, apply the neural network to pairs of portions of the 3D modeled object, thereby outputting a set of values, each for a respective pair of portions of the 3D modeled object, each indicating an extent to which the two portions belong to a same segment of the 3D modeled object, and
perform a segmentation of the 3D modeled object based on a set of one or more similarity indicators between neighboring portions of the 3D modeled object, the set of similarity indicators including the set of values,
wherein the processor is further configured to learn the neural network by being configured to minimize a loss comprising a term penalizing, for pairs of the dataset, a disparity between a respective value that indicates whether or not the two portions of a pair belong to a same segment of a 3D modeled object and a second respective value output by the neural network for the two portions,
wherein the neural network transforms the input two portions each into a corresponding intermediary feature, and the loss further includes another term penalizing, for said pairs of the dataset, a disparity between the respective value that indicates whether or not the two portions of a pair belong to a same segment of a 3D modeled object and a proximity between the intermediary features corresponding to the two portions, and
wherein the term is of a type:
∑
i
=
1
N
d
CE
(
g
w
(
f
w
(
X
i
(
1
)
)
,
f
w
(
X
i
(
2
)
)
)
,
p
i
)
,
where ((X 1 (1) , X 2 (2) ), . . . (X N (1) , X N (2) )) are pairs of the dataset, g w (f w (X i (1) ), f w (X i (2) )) where ((x ( 1 ), x ( 2 )) . . . is the respective value outputted by the neural network for the pair (X i (1) , X i (2) ), g w ∘f w is the neural network, p i is the respective value that indicates whether or not portions X i (1) and X i (2) belong to a same segment of a 3D modeled object, and d CE is a loss penalizing a disparity between g w (f w (X i (1) ), f w (X i (2) )) and p i ,
and the other term is of the type:
∑
i
=
1
N
d
M
(
f
w
(
X
i
(
1
)
)
,
f
w
(
X
i
(
2
)
)
,
p
i
)
,
where, f w (X i (1) ) and f w (X i (2) ) are the intermediary features corresponding to portions X i (1) and X i (2) of the pair (X i (1) , X i (2) ), and d M is a loss penalizing a disparity between p i and a proximity between f w (X i (1) ) and f w (X i (2) ).
10. The device of claim 9 , wherein the neural network includes:
an encoding neural network configured to take as input a portion of a 3D modeled object and for encoding the portion into a latent vector, and
a predictive neural network configured to take as input a pair of latent vectors outputted by the encoding neural network and to output a respective value that indicates an extent to which the two portions encoded by the latent vectors belong to a same segment of the 3D modeled object.
11. The device of claim 10 , wherein the encoding neural network is configured to take as input a portion of a 3D modeled object and neighboring portions of the input portion.
12. The device of claim 9 , wherein the processor is further configured to obtain the dataset by being further configured to:
obtain an initial dataset of segmented 3D modeled objects, and
for each segmented 3D modeled object of the initial dataset, generate pairs of portions of the segmented 3D modeled object, the portions of a pair belonging either to a same segment of the segmented 3D modeled object or to different segments, each pair including a respective value that indicates whether or not the two portions belong to a same segment of a 3D modeled object.
13. The device of claim 12 , wherein for one or more segmented 3D modeled objects of the initial dataset, the processor is further configured to generate the pairs by being configured to generate pairs of portions for which at least one portion does not belong entirely to a single segment.
14. The device of claim 12 , wherein the processor is further configured to obtain the dataset by being configured to create mini-batches of pairs, the learning being carried out mini-batch by mini-batch.