Representation learning for continuous vector graphics
Systems and methods for generating representations for vector graphics are described. Embodiments are configured to obtain semantic information and geometric information for a vector graphics image. The semantic information describes individual segments of the vector graphics image, and the geometric information describes geometric relationships among the individual segments. Embodiments are additionally configured to encode the semantic information and the geometric information to obtain a vector graphics representation for the vector graphics image, and to provide a reconstructed image based on the vector graphics representation.
1 . A method comprising:
obtaining semantic information and geometric information for a vector graphics image, wherein the semantic information describes individual segments of the vector graphics image, and the geometric information describes geometric relationships among the individual segments;
generating an adjacency matrix based on the geometric information;
encoding the semantic information and the geometric information by performing geometric self-attention based on the semantic information and the adjacency matrix to obtain a vector graphics representation for the vector graphics image; and
providing a reconstructed image based on the vector graphics representation.
2 . The method of claim 1 , further comprising:
embedding the semantic information and the geometric information to obtain a combined embedding, wherein the vector graphics representation is based on the combined embedding.
3 . The method of claim 1 , further comprising:
performing a medial axis transform (MAT) on the semantic information to obtain the geometric information.
4 . The method of claim 1 , further comprising: generating a graph based on the semantic information and the adjacency matrix; and applying a graph convolution network (GCN) to the graph, wherein the geometric self-attention is based on the GCN.
5 . The method of claim 1 , further comprising:
decoding the vector graphics representation to obtain decoded semantic information; and
generating the reconstructed image based on the decoded semantic information.
6 . The method of claim 1 , further comprising:
classifying the vector graphics image based on the vector graphics representation to obtain an image type, wherein the reconstructed image is provided based on the image type.
7 . The method of claim 1 , further comprising:
generating an additional vector graphics representation based on an additional vector graphics image;
interpolating between the vector graphics representation and the additional vector graphics representation to obtain an interpolated vector graphics representation; and
decoding the interpolated vector graphics representation to obtain interpolated semantic information, wherein the reconstructed image is based on the interpolated semantic information.
8 . The method of claim 1 , further comprising:
obtaining a query vector graphics image;
encoding semantic information and geometric information from the query vector graphics image to obtain a query vector graphics representation;
comparing the vector graphics representation and the query vector graphics representation; and
providing the reconstructed image based on the comparison.
9 . A method comprising:
obtaining semantic information and geometric information for a vector graphics image, wherein the semantic information describes individual segments of the vector graphics image, and the geometric information describes geometric relationships among the individual segments;
encoding, using a machine learning model, the semantic information and the geometric information to obtain a vector graphics representation for the vector graphics image;
decoding, using the machine learning model, the vector graphics representation to obtain decoded semantic information;
comparing the semantic information and the decoded semantic information; and
updating parameters of the machine learning model based on the comparison.
10 . The method of claim 9 , further comprising:
generating a reconstruction loss based on the comparison, where the reconstruction loss measures a difference between original commands included in the semantic information and reconstructed commands included in the decoded semantic information; and
updating parameters of the machine learning model based on the reconstruction loss.
11 . The method of claim 9 , further comprising:
generating a classification loss based on the comparison, where the classification loss measures a difference between command classifications included in the semantic information and reconstructed command classifications included in the decoded semantic information; and
updating parameters of the machine learning model based on the classification loss.
12 . The method of claim 11 , wherein:
the command classifications in both the semantic information and the decoded semantic information belong to a set of command types, and wherein the set of command types includes a “Move” command, a “Line” command, and a “Curve” command.
13 . An apparatus comprising:
a processor;
a memory including instructions executable by the processor;
a vector graphics encoder configured to encode semantic information and geometric information of a vector graphics image to obtain a vector graphics representation for the vector graphics image; and
a decoder configured to decode the vector graphics representation into a list of reconstructed commands.
14 . The apparatus of claim 13 , further comprising:
a medial axis transform (MAT) component configured to perform MAT on the semantic information to obtain the geometric information.
15 . The apparatus of claim 13 , further comprising:
an embedding component configured to embed the semantic information and the geometric information to obtain a combined embedding.
16 . The apparatus of claim 15 , wherein:
the embedding component is further configured to generate an adjacency matrix.
17 . The apparatus of claim 13 , wherein:
the vector graphics encoder comprises a geometric self-attention component including a graph convolution network (GCN), and wherein the geometric self-attention component is configured to adjust the vector graphics representation using the GCN.