Recognition of a 3D modeled object from a 2D image
The invention notably relates to a computer-implemented method for recognizing a three-dimensional modeled object from a two-dimensional image. The method comprises providing a first set of two-dimensional images rendered from three-dimensional modeled objects, each two-dimensional image of the first set being associated to a label; providing a second set of two-dimensional images not rendered from three-dimensional objects, each two-dimensional image of the second set being associated to a label; training a model on both first and second sets; providing a similarity metric; submitting a two-dimensional image depicting at least one object; and retrieving a three-dimensional object similar to the said at least one object of the two-dimensional image submitted by using the trained model and the similarity metric.
1. A computer-implemented method for recognizing a three-dimensional modeled object from a two-dimensional image, comprising:
obtaining a first set of two-dimensional images rendered from three-dimensional modeled objects, each two-dimensional image of the first set being computed from the three-dimensional modeled objects and being associated with a label before being obtained;
obtaining a second set of two-dimensional images not rendered from three-dimensional objects, each two-dimensional image of the second set being associated to a label;
training a model on both first and second sets;
obtaining a similarity metric;
submitting a two-dimensional image depicting at least one object; and
retrieving a three-dimensional object similar to the said at least one object of the two-dimensional image submitted by using the trained model and the similarity metric.
2. The computer-implemented method of claim 1 , wherein each two-dimensional image of the first set is computed from a viewpoint on a three-dimensional object, the viewpoint being selected among a plurality of viewpoints on the three-dimensional object.
3. The computer-implemented method of claim 2 , wherein the plurality of viewpoints on the three-dimensional object is obtained from a Thomson sphere.
4. The computer-implemented method of claim 1 , further comprising, after training the model:
building an index of the two-dimensional images of the first set by extracting a feature vector for each two-dimensional image of the first set, wherein a feature vector is extracted using the trained model.
5. The computer-implemented method of claim 4 , wherein an extracted feature vector comprises successive applications of parameters of the trained model to a two-dimensional image.
6. The computer-implemented method of claim 4 , further comprising:
extracting a feature vector of the submitted two-dimensional image.
7. The computer-implemented method of claim 6 , further comprising:
comparing the extracted feature vector of the submitted two-dimensional image with the indexed feature vectors by using the similarity metric.
8. The computer-implemented method of claim 7 , wherein the similarity metric used for the comparison is deterministic.
9. The computer-implemented method of claim 7 , wherein the similarity metric used for the comparison is learned with a learning process.
10. The computer-implemented method of claim 9 , wherein the learning process comprises:
training a similarity model on both first and second sets, each two-dimensional image of the first set being paired with a two-dimensional image of the second set and the labels associated with two-dimensional images of the first and second set comprising at least a similarity information label.
11. The computer-implemented method of claim 9 , wherein the extraction of the feature vector and the learning process of the similarity metric are concomitantly carried out by using a Siamese network.
12. The computer-implemented method of claim 1 , wherein the trained model is obtained with a Deep Neural Network.
13. A non-transitory computer readable medium having stored thereon a computer program for recognizing a three-dimensional modeled object from a two-dimensional image, comprising instruction causing processing circuitry to perform the method comprising:
obtaining a first set of two-dimensional images rendered from three-dimensional modeled objects, each two-dimensional image of the first set being computed from the three-dimensional modeled objects and being associated with a label before being obtained;
obtaining a second set of two-dimensional images not rendered from three-dimensional objects, each two-dimensional image of the second set being associated to a label;
training a model on both first and second sets;
obtaining a similarity metric;
submitting a two-dimensional image depicting at least one object; and
retrieving a three-dimensional object similar to the said at least one object of the two-dimensional image submitted by using the trained model and the similarity metric.
14. A system comprising a processing circuitry coupled to a non-transitory memory, the non-transitory memory having recorded thereon the computer program of claim 13 .
15. The computer-implemented method of claim 1 , wherein each two-dimensional image of the first set is computed from a viewpoint on a three-dimensional object, the viewpoint being selected among a plurality of viewpoints on the three-dimensional object,
wherein the plurality of viewpoints on the three-dimensional object is obtained from a Thomson sphere, and
wherein the method further comprises, after training the model:
building an index of the two-dimensional images of the first set by extracting a feature vector for each two-dimensional image of the first set, wherein a feature vector is extracted using the trained model.