Granted Patent B2

US 12,210,813 · App. 18/051,720 · Granted Jan 28, 2025

Inventors: Pranay Kumar (Noida, IN); Nipun Jindal (Delhi, IN)

Assignee: Adobe Inc.

G06F40/109

Abstract

Claims (77)

1. A non-transitory computer-readable medium storing executable instructions which, when executed by at least one processing device, cause the at least one processing device to perform operations comprising:

identifying, utilizing a multi-modal font machine-learning model comprising a glyph shape-aware machine learning model and a glyph metrics analyzer to process data in a unified latent space, a recommended font corresponding to a source font by:

receiving, from a computing device, an indication of the source font;

generating, utilizing an outline sequence encoders of the multi-modal font machine-learning model, a sequence embedding representing an outline for a reference glyph of the source font from sequence aspect features;

generating, utilizing a local point encoder of the multi-modal font machine-learning model, a visual structure embedding from shape parameters of the reference glyph representing local points of the reference glyph;

generating, utilizing the glyph metrics analyzer of the multi-modal font machine-learning model, a glyph metrics embedding for measurements of the reference glyph;

generating, utilizing the multi-modal font machine-learning model, a multi-modal vector representing the source font by combining the sequence embedding, the visual structure embedding, and the glyph metrics embedding;

comparing, in the unified latent space, the multi-modal vector indicating the sequence embedding, the visual structure embedding, and the glyph metrics embedding of the source font with candidate multi-modal vectors representing candidate fonts;

identifying, utilizing inference layers in the unified latent space of the multi-modal font machine-learning model to process the multi-modal vector, the recommended font from a plurality of candidate fonts that corresponds to the source font.

2. The non-transitory computer-readable medium of claim 1 , wherein generating the sequence embedding comprises:

utilizing the outline sequence encoder to process the sequence aspect features comprising a rasterized image of the reference glyph and a vector outline of the reference glyph; and

based on processing the sequence aspect features, generating the sequence embedding.

3. The non-transitory computer-readable medium of claim 1 , wherein utilizing the inference layers in the unified latent space of the multi-modal font machine-learning model comprises:

mapping the multi-modal vector in the unified latent space and the candidate multi-modal vectors representing the candidate fonts in the unified latent space;

determining, utilizing fully connected layers of a feed forward neural network, measures of cosine similarity between the multi-modal vector and the candidate multi-modal vectors representing the candidate fonts; and

generating, utilizing a softmax layer of the multi-modal font machine-learning model, a distribution of font probabilities relating to the measures of cosine similarity between the multi-modal vector and the candidate multi-modal vectors representing the candidate fonts.

4. The non-transitory computer-readable medium of claim 1 , wherein generating the visual structure embedding representing the local points of the reference glyph further comprises generating, utilizing a local point encoder, a vector representing local structure points and comprising invariant features of the reference glyph based on a rasterized image of the reference glyph.

5. The non-transitory computer-readable medium of claim 1 , wherein generating the glyph metrics embedding for measurements of the reference glyph further comprises:

determining, utilizing the glyph metrics analyzer, glyph metrics of the reference glyph; and

normalizing, utilizing a metrics normalizer, the glyph metrics based on a distribution of glyph metrics.

6. The non-transitory computer-readable medium of claim 5 , wherein generating the glyph metrics further comprises determining at least one of a glyph width, a glyph height, a glyph ascender metric, a glyph descender metric, a lower-case height, an upper-case height, a stem width, an average glyph contrast, or a stem angle.

7. The non-transitory computer-readable medium of claim 1 , wherein generating the multi-modal vector representing the source font further comprises concatenating the sequence embedding, the visual structure embedding, and the glyph metrics embedding.

8. The non-transitory computer-readable medium of claim 1 , wherein generating the multi-modal vector comprises concatenating, utilizing a multi-layer perceptron, a raster embedding generated from the reference glyph of the source font, the sequence embedding, the visual structure embedding, and the glyph metrics embedding.

9. The non-transitory computer-readable medium of claim 1 , wherein identifying the recommended font from the plurality of candidate fonts that corresponds to the source font further comprises:

identifying natural language processing (NLP) tags classifying glyphs of candidate fonts;

determining a subset of candidate fonts of the candidate fonts based on the NLP tags; and

identifying, from the subset of candidate fonts, the recommended font corresponding to the source font based on the multi-modal vector, wherein the multi-modal vector comprises a concatenation of a raster embedding generated from the reference glyph of the source font, the sequence embedding, the visual structure embedding, and the glyph metrics embedding.

10. The non-transitory computer-readable medium of claim 1 , wherein identifying the recommended font corresponding to the source font based on the multi-modal vector further comprises:

determining, utilizing a cosine similarity model, cosine similarities between the multi-modal vector and the candidate multi-modal vectors representing the candidate fonts;

generating, utilizing a softmax layer of the multi-modal font machine-learning model, a distribution of font probabilities relating to measures of cosine similarity between the multi-modal vector and the candidate multi-modal vectors; and

determining, from among the candidate multi-modal vectors, a candidate multi-modal vector for the recommended font corresponding to a highest cosine similarity of the measures of cosine similarity.

11. A system comprising:

at least one memory device comprising font files corresponding to candidate fonts; and

at least one processor configured to cause the system to:

identify, utilizing a multi-modal font machine-learning model comprising a glyph shape-aware machine learning model and a glyph metrics analyzer to process data in a unified latent space, a recommended font corresponding to a source font by:

receiving, from a computing device, an indication of the source font;

generating, utilizing an outline sequence encoders of the multi-modal font machine-learning model, a sequence embedding representing an outline for at least a reference glyph of the source font from sequence aspect features;

generating, utilizing a local point encoder of the multi-modal font machine-learning model, a visual structure embedding for at least the reference glyph of the source font from shape parameters of at least the reference glyph;

generating, utilizing the glyph metrics analyzer of the multi-modal font machine-learning model, a glyph metrics embedding for measurements of the reference glyph;

generating, utilizing a multi-layer perceptron of the multi-modal font machine-learning model, a multi-modal vector representing at least the reference glyph by combining the sequence embedding, the visual structure embedding, and the glyph metrics embedding;

comparing, in the unified latent space, the multi-modal vector indicating the sequence embedding, the visual structure embedding, and the glyph metrics embedding of the source font with candidate multi-modal vectors representing candidate fonts; and

identifying, utilizing inference layers of the multi-modal font machine-learning model to process the multi-modal vector, the recommended font corresponding to the source font.

12. The system of claim 11 , further comprises generating the sequence embedding, the visual structure embedding, and the glyph metrics embedding for multiple reference glyphs of the source font.

13. The system of claim 11 , wherein generating the visual structure embedding further comprises:

extracting, from at least a rasterized image of at least the reference glyph, one or more local points; and

generating, utilizing the local point encoder, a scale-invariant feature transform (SIFT) vector representing local structure points of at least the reference glyph.

14. The system of claim 11 , wherein generating the glyph metrics embedding for measurements of at least the reference glyph further comprises:

generating, utilizing the glyph metrics analyzer, glyph metrics of at least the reference glyph;

processing dimensions for the glyph metrics of at least the reference glyph; and

normalizing, by utilizing a glyph metric normalizer, the processed dimensions for the glyph metrics of at least the reference glyph.

15. The system of claim 11 , generating the multi-modal vector representing the source font further comprises concatenating, utilizing a multi-modality blender algorithm, the sequence embedding, the visual structure embedding, and the glyph metrics embedding.

16. The system of claim 11 , wherein identifying the recommended font corresponding to the source font based on the multi-modal vector further comprises:

identifying natural language processing (NLP) tags classifying glyphs of the candidate fonts;

removing a subset of candidate fonts of the candidate fonts based on the NLP tags; and

identifying, from a remaining subset of candidate fonts of the candidate fonts, the recommended font corresponding to the source font based on the multi-modal vector.

17. The system of claim 11 , wherein identifying the recommended font corresponding to the source font based on the multi-modal vector further comprises:

determining a final multi-modal vector representing the source font by:

generating a raster embedding representing a rasterized image of at least the reference glyph;

concatenating the sequence embedding, the visual structure embedding, the glyph metrics embedding, and the raster embedding; and

identifying the recommended font from the multi-modal vector comprising the concatenation of the sequence embedding, the visual structure embedding, the glyph metrics embedding, and the raster embedding.

18. A method comprising:

identifying, utilizing a multi-modal font machine-learning model comprising a glyph shape-aware machine learning model and a glyph metrics analyzer to process data in a unified latent space, a recommended font corresponding to a source font by:

receiving, from a computing device, an indication of the source font;

generating, utilizing an outline sequence encoders of the multi-modal font machine-learning model, a sequence embedding representing an outline for at least a reference glyph of the source font from sequence aspect features;

generating, utilizing a local point encoder of the multi-modal font machine-learning model, a visual structure embedding for at least the reference glyph of the source font from shape parameters of the reference glyph;

generating, utilizing the glyph metrics analyzer of the multi-modal font machine-learning model, a glyph metrics embedding for measurements of at least the reference glyph;

generating, utilizing the multi-modal font machine-learning model, a multi-modal vector representing the source font by combining the sequence embedding, the visual structure embedding, and the glyph metrics embedding;

comparing, in the unified latent space, the multi-modal vector indicating the sequence embedding, the visual structure embedding, and the glyph metrics embedding of the source font with candidate multi-modal vectors representing candidate fonts; and

identifying, utilizing inference layers of the multi-modal font machine-learning model to process the multi-modal vector, the recommended font corresponding to the source font.

19. The method of claim 18 , further comprising training a convolutional neural network and a transformer model part of a glyph shape-aware machine learning model of the multi-modal font machine-learning model by:

receiving input glyphs from a font;

processing, utilizing the glyph shape-aware machine learning model, Bezier curves of same font styles for the font and raster versions generated from the Bezier curves;

learning parameters of sequence for the font and rasterization data for the font for the glyph shape-aware machine learning of the multi-modal font machine learning model based on processing the Bezier curves and the raster versions.

20. The method of claim 18 , wherein generating the glyph metrics embedding for measurements of at least the reference glyph further comprises:

determining, from metadata for multiple reference glyphs, glyph metrics of the multiple reference glyphs;

processing the glyph metrics of the multiple reference glyphs for horizontal, vertical, and diagonal dimensions; and

normalizing, by utilizing a glyph metric normalizer, the glyph metrics based on a distribution of glyph metrics.

Assignments (1)

Continuity (1)

References Cited (89)