Systems and methods for providing supplemental content based on a meme

1 . A computer-implemented method comprising:

identifying a first meme for presentation at a display of a computing device;

generating a data structure comprising data associated with a plurality of memes, wherein the plurality of memes comprise a plurality of root memes, each respective root meme of the plurality of root memes being associated with a plurality of variant memes being variations of the respective root meme, and wherein the data indicates, for each respective meme of the plurality of memes, at least one category, of a plurality of categories, associated with visual content of the respective meme, and wherein the generating comprises:

storing, as data of the data structure, an indication that a second meme of the plurality of memes belongs to a first category of the plurality of categories, based at least in part on using a first machine learning model to receive as input one or more tokens comprising one or more embeddings representing natural language descriptions associated with visual content of the second meme and to output data indicative of the first category, wherein the natural language descriptions associated with visual content of the second meme are generated using a second machine learning model; and

learning, by the first machine learning model, that a first embedding representing at least a portion of the second meme based at least in part on the one or more natural language descriptions associated with the visual content of the second meme is highly determinative of categorization of memes into the first category, wherein the first embedding is one of a plurality of embeddings representing at least a portion of the second meme, and wherein the learning comprises training the first machine learning model using a subset of embedding data corresponding to memes of the plurality of memes previously categorized in the first category and calculating values indicative of an amount of alignment between embeddings and categorization into the first category;

determining that a category to which visual content of the first meme belongs corresponds to the first category of the second meme associated with the data stored in the data structure, wherein the determining comprises:

extracting a second embedding representing at least a portion of the first meme, based at least in part on one or more natural language descriptions associated with visual content of the first meme generated using the second machine learning model;

querying, via an application programing interface (API), the generated data structure with the extracted second embedding;

determining, via an API returned result determined based at least in part on a comparison between the extracted second embedding and the first embedding representing at least a portion of the second meme learned to be highly correlated with categorization of memes into the first category, that the extracted second embedding representing the at least a portion of the first meme corresponds to the first category; and

based at least in part on determining the extracted second embedding corresponds to the first category, assigning the first meme to the first category, without having to fully execute the first machine learning model to output an indication of a category to which the first meme belongs;

determining, based at least in part on a number of variant memes associated with the second meme in the data structure, whether a popularity of the second meme exceeds a threshold; and

in response to determining that the popularity of the second meme exceeds the threshold:

identifying supplemental content related to the first category assigned to the first meme and the second meme; and

causing the supplemental content to be provided for presentation in a vicinity of the first meme at the display of the computing device.

2 . The method of claim 1 , wherein determining that the category to which the visual content of the first meme corresponds to the first category of the second meme associated with the data stored in the data structure further comprises determining that the first meme is a variant meme of the second meme.

3 . The method of claim 1 , wherein a particular image and images associated with the particular image are added to the data structure as a root meme and variant memes, respectively, based on determining that the images associated with the particular image correspond to at least a threshold number of user-generated variations of the particular image.

4 . The method of claim 1 , further comprising:

determining a particular variant meme for a particular root meme stored in the data structure by:

identifying a modified version of the particular root meme having a modification to at least one visual element of the particular root meme; and

calculating a similarity score between the particular root meme and the modified version of the meme;

in response to determining that the similarity score is above a threshold, determining that the modified version of the particular root meme is a variant meme of the particular root meme; and

storing the particular variant meme in the data structure in association with the particular root meme.

5 . The method of claim 1 , wherein the data structure comprises a graph data structure, and a distance between a particular root meme and a particular variant meme of the plurality of memes in the graph data structure is indicative of a degree of similarity between a category of the particular root meme and a category of the particular variant meme.

6 . The method of claim 5 , wherein the graph data structure comprises a plurality of graph data structures, and wherein each respective category of the plurality of categories is associated with the respective graph data structure of the plurality of graph data structures that is organized based at least in part on popularities of memes associated with the respective category.

7 . The method of claim 6 , wherein identifying the supplemental content related to the first category assigned to the first meme and the second meme, and causing the supplemental content to be provided for presentation in the vicinity of the first meme at the display of the computing device, is further performed based at least in part on a distance between the graph data structure for the first category of the first meme and one or more other graph data structures, in the plurality of graph data structures, for one or more other categories of memes.

8 . The method of claim 1 , further comprising:

receiving an indication that the supplemental content should not be provided in association with a meme of a particular category; and

wherein causing the supplemental content to be provided for presentation in the vicinity of the first meme at the display of the computing device is performed in response to determining that the first category of the first meme does not correspond to the particular category.

9 . The method of claim 1 , wherein a third meme is presented for display at the computing device, the method further comprises:

determining that a category to which visual content of the third meme does not belong corresponds to the first category of the visual content of the second meme associated with the data stored in the data structure, wherein the determining comprises:

extracting a third embedding representing at least a portion of the third meme, based at least in part on one or more natural language descriptions associated with visual content of the third meme;

querying, via the API, the data structure with the extracted third embedding;

determining, via a second API returned result, whether the extracted third embedding representing the at least a portion of the third meme corresponds to the first category; and

based at least in part on determining the extracted embedding does not correspond to the first category based on the second API result, executing the first machine learning model to output an indication of a category to which the third meme belongs.

10 . The method of claim 9 , the method further comprising: training the first machine learning model to learn a plurality of embeddings for tokens associated with indications of characteristics of a dataset of memes, wherein the characteristics of the dataset of memes are obtained using the second machine learning model.

11 . The method of claim 10 , further comprising:

determining that a particular embedding, of the plurality of embeddings, for a particular token is used by the first machine learning model to output data indicative of the particular category; and

based on determining that the particular embedding for the particular token is used by the first machine learning model to output data indicative of the particular category, using the particular token to identify one or more variant memes for one or more root memes.

12 . The method of claim 10 , further comprising:

using the first machine learning model to determine that the third meme is a variant meme of the first meme, and that the third meme belongs to a different category than the particular category of the first meme;

identifying a characteristic of the third meme that caused the third meme to be classified as belonging to the different category; and

assigning, in the first machine learning model, a relatively higher weighting in association with an embedding for a token corresponding to the characteristic of the third meme.

13 . The method of claim 1 , wherein identifying the supplemental content related to the first category assigned to the first meme and the second meme comprises:

transmitting, to a server, an indication of the first category of the first meme; and

receiving, from the server, the supplemental content, wherein the server identifies the supplemental content from among a plurality of supplemental content items based on determining that the supplemental content depicts an object having a characteristic that matches the first category of the first meme.

14 . The method of claim 1 , further comprising:

identifying a demographic group associated with the first meme;

identifying preferences of a user of the computing device;

determining, based on the preferences of the user, a particular demographic group, of a plurality of demographic groups, to which the user belongs;

determining the particular demographic group of the user corresponds to the demographic group associated with the first meme; and

wherein causing the supplemental content to be provided for presentation in the vicinity of the first meme at the display of the computing device is performed in response to determining that the particular demographic group of the user corresponds to the demographic group associated with the first meme.

15 . The method of claim 1 , further comprising:

determining that an activity level associated with the root meme and the associated plurality of variant memes is below a threshold; and

in response to determining that activity associated with the root meme and the associated plurality of variant memes is below the threshold, deleting the root meme and the associated plurality of variant memes from the data structure.

16 . The method of claim 1 , wherein the data structure comprising data associated with the plurality of memes further comprises data indicative of an activity level associated with the plurality of memes.

17 . The method of claim 1 , wherein the first meme is being displayed via a social media platform, and the method further comprises:

based on identifying a plurality of users of the social media platform having user preferences related to the first category of first meme, causing the supplemental content to be provided for presentation in the vicinity of the first meme at the display of a plurality of computing devices of the plurality of users of the social media platform.

18 . The method of claim 1 , wherein:

the data structure comprises a graph data structure, and a distance between a particular root meme and a particular variant meme of the plurality of memes in the graph data structure is indicative of a degree of similarity between a category of the particular root meme and a category of the particular variant meme; and

the method further comprises determining a user's degree of association with a category based on a distance between a particular root meme and a particular variant meme of the plurality of memes indicated in the graph data structure.

19 . A system comprising:

computer memory; and

control circuitry configured to:

identify a first meme for presentation at a display of a computing device;

generate a data structure comprising data associated with a plurality of memes, wherein the plurality of memes comprise a plurality of root memes, each respective root meme of the plurality of root memes being associated with a plurality of variant memes being variations of the respective root meme, and wherein the data indicates, for each respective meme of the plurality of memes, at least one category, of a plurality of categories, associated with visual content of the respective meme, and wherein the generating comprises:

store, as data of the data structure, an indication that a second meme of the plurality of memes belongs to a first category of the plurality of categories, based at least in part on using a first machine learning model to receive as input one or more tokens comprising one or more embeddings representing natural language descriptions associated with visual content of the second meme and to output data indicative of the first category, wherein the natural language descriptions associated with visual content of the second meme are generated using a second machine learning model; and

learn, by the first machine learning model, that a first embedding representing at least a portion of the second meme based at least in part on the one or more natural language descriptions associated with the visual content of the second meme is highly determinative of categorization of memes into the first category, wherein the first embedding is one of a plurality of embeddings representing at least a portion of the second meme, and wherein the learning comprises training the first machine learning model using a subset of embedding data corresponding to memes of the plurality of memes previously categorized in the first category and calculating values indicative of an amount of alignment between embeddings and categorization into the first category;

determine that a category to which visual content of the first meme belongs corresponds to the first category of the second meme associated with the data stored in the data structure, wherein the determining comprises:

extract a second embedding representing at least a portion of the first meme, based at least in part on one or more natural language descriptions associated with visual content of the first meme generated using the second machine learning model;

query, via an application programing interface (API), the generated data structure with the extracted second embedding;

determine, via an API returned result determined based at least in part on a comparison between the extracted second embedding and the first embedding representing at least a portion of the second meme learned to be highly correlated with categorization of memes into the first category, that the extracted second embedding representing the at least a portion of the first meme corresponds to the first category; and

based at least in part on determining the extracted second embedding corresponds to the first category, assign the first meme to the first category, without having to fully execute the first machine learning model to output an indication of a category to which the first meme belongs;

determine, based at least in part on a number of variant memes associated with the second meme in the data structure, whether a popularity of the second meme exceeds a threshold; and

in response to determining that the popularity of the second meme exceeds the threshold:

identify supplemental content related to the first category assigned to the first meme and the second meme; and

cause the supplemental content to be provided for presentation in a vicinity of the first meme at the display of the computing device.

20 . The system of claim 19 , wherein the control circuitry is configured to determine that the category to which the visual content of the first meme corresponds to the first category of the second meme associated with the data stored in the data structure by determining that the first meme is a variant meme of the second meme.