Machine learning-based inference of granular font properties
A textual properties model is used to infer values for certain font properties of interest given certain text-related data, such as rendered text images. The model may be used for numerous purposes, such as aiding with document layout, identifying font families that are similar to a given font families, and generating new font families with specific desired properties. In some embodiments, the model is trained from a combination of synthetic data that is labeled with values for the font properties of interest, and partially-labeled data from existing “real-world” documents.
1. A computer-implemented method for inference of font properties based on rendered text, the computer-implemented method comprising:
obtaining a rendered text image of text in a document;
training a machine learning model to infer values for a plurality of font properties based on rendered text images, the training using a training set comprising a plurality of partially-labeled rendered text images; and
using the machine learning model to infer values for font properties of a font family of the rendered text;
wherein the partially-labeled rendered text images specify values for less than all of the plurality of font properties.
2. The computer-implemented method of claim 1 , further comprising obtaining the rendered text image from a web browser textual rendering of text on a web site.
3. The computer-implemented method of claim 1 , wherein the font properties for which values are inferred comprise one or more of: horizontal character metrics, vertical character metrics, character pixel density, italic angle, serif type, or character bounding box.
4. The computer-implemented method of claim 1 , further comprising training the machine learning model as a convolutional neural network having custom twistor convolutional layers substantially providing textual rotational invariance.
5. The computer-implemented method of claim 1 , where the training set additionally comprises a plurality of synthetic rendered text images.
6. The computer-implemented method of claim 5 , further comprising generating the synthetic rendered text images, the generating comprising:
generating support vector graphic (SVG) data for text; and
deriving values for the plurality of font properties through image analysis of renderings of SVG;
wherein training the machine learning model uses the derived values for the plurality of font properties as training labels for the synthetic rendered text images.
7. The computer-implemented method of claim 1 , further comprising:
performing unsupervised pre-training on unlabeled rendered text images;
priming the machine learning model using results of the pre-training; and
training the primed machine learning model.
8. The computer-implemented method of claim 1 , further comprising:
using the inferred values for the font properties to identify a different font family that is visually similar to the font family of the rendered text.
9. The computer-implemented method of claim 1 , further comprising:
using the machine learning model to infer values for font properties of font families of a plurality of additional rendered text images; and
using the inferred values for the font properties of the font family of the rendered text image, and of the inferred values of the font properties of the font families of the plurality of additional rendered text images, to generate a new font family.
10. A computer-implemented method for inference of font properties based on rendered text, the computer-implemented method comprising:
generating a plurality of synthetic rendered text images, the generating comprising:
generating support vector graphic (SVG) data for text in a font family; and
deriving values for a plurality of font properties of the font family through image analysis of renderings of SVG;
training a machine learning model to infer values for the a plurality of font properties based on rendered text images, the training using a training set comprising the plurality of synthetic rendered text images; and
obtaining a rendered text image of text in a document; and
using the machine learning model to infer values for the font properties of the rendered text;
wherein training the machine learning model uses the derived values for the plurality of font properties as training labels for the synthetic rendered text images.
11. The computer-implemented method of claim 10 , further comprising obtaining the rendered text image from a web browser textual rendering of text on a web site.
12. The computer-implemented method of claim 10 , wherein the font properties for which values are inferred comprise one or more of: horizontal character metrics, vertical character metrics, character pixel density, italic angle, serif type, or character bounding box.
13. The computer-implemented method of claim 10 , further comprising training the machine learning model as a convolutional neural network having custom twistor convolutional layers substantially providing textual rotational invariance.
14. The computer-implemented method of claim 10 , wherein the training comprises using a training set comprising a plurality of partially-labeled rendered text images and a plurality of synthetic rendered text images.
15. The computer-implemented method of claim 10 , further comprising:
performing unsupervised pre-training on unlabeled rendered text images;
priming the machine learning model using results of the pre-training; and
training the primed machine learning model.
16. The computer-implemented method of claim 10 , further comprising:
using the inferred values for the font properties to identify a different font family that is visually similar to the font family of the rendered text.
17. A computer-implemented method for inference of font properties based on rendered text, the computer-implemented method comprising:
obtaining a rendered text image of text in a document;
obtaining a machine learning model trained to infer values for a plurality of font properties based on rendered text images; and
using the machine learning model to infer values for font properties of a font family of the rendered text;
using the machine learning model to infer values for font properties of font families of a plurality of additional rendered text images; and
using the inferred values for the font properties of the font family of the rendered text image, and of the inferred values of the font properties of the font families of the plurality of additional rendered text images, to generate a new font family.
18. The computer-implemented method of claim 17 , further comprising obtaining the rendered text image from a web browser textual rendering of text on a web site.
19. The computer-implemented method of claim 17 , wherein the font properties for which values are inferred comprise one or more of: horizontal character metrics, vertical character metrics, character pixel density, italic angle, serif type, or character bounding box.
20. The computer-implemented method of claim 17 , further comprising training the machine learning model as a convolutional neural network having custom twistor convolutional layers substantially providing textual rotational invariance.
21. The computer-implemented method of claim 17 , wherein the training comprises using a training set comprising a plurality of partially-labeled rendered text images and a plurality of synthetic rendered text images.
22. The computer-implemented method of claim 17 , further comprising:
performing unsupervised pre-training on unlabeled rendered text images;
priming the machine learning model using results of the pre-training; and
training the primed machine learning model.
23. The computer-implemented method of claim 17 , further comprising:
using the inferred values for the font properties to identify a different font family that is visually similar to the font family of the rendered text.