IP Library Granted Patent US 12,387,404
Granted Patent B2
US 12,387,404 · App. 18/219,473 · Granted Aug 12, 2025

Generating memes and enhanced content in electronic communication

Inventors: Ning Xu (Irvine, CA); Serhad Doken (Bryn Mawr, PA); Reda Harb (Tampa, FL)
Assignee: Adeia Guides Inc.
G06T11/60G06F3/0482G06F40/30H04L51/10G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,387,404
App. No.
18/219,473
Granted
Aug 12, 2025
Kind
B2
Abstract

Methods and systems are described for generating memes and enhanced content in electronic communication. The communication is received, analyzed, and assessed for determining whether the enhanced content should be included in the communication. A content, context, intent, and/or mood of the communication is determined. For meme generation implementations, images and captions are generated based on the communication. Some candidate memes incorporate an image of one of the participants in the communication. Enhanced content candidates are generated and presented to the user for inclusion in the communication. Artificial intelligence systems, including neural networks, and models are utilized to improve the enhanced content generation. Related apparatuses, devices, techniques, and articles are also described.

Claims (51)

1. A method comprising:

accessing a portion of an electronic communication, wherein the accessing the portion of the electronic communication includes determining a context of the portion of the electronic communication;

determining a textual response and a visual response to the portion of the electronic communication;

determining whether a meme suggestion mode should be activated based on the electronic communication;

generating a candidate meme comprising an image including the textual response and the visual response;

providing for display as part of the electronic communication a selectable option including display of the candidate meme; and

in response to selection of the selectable option, providing the candidate meme for display within the electronic communication, wherein the textual response is based at least in part on the context determined from the portion of the electronic communication.

2. The method of claim 1 , comprising:

in response to determining the meme suggestion mode should be activated, executing the determining, the generating, the providing of the selectable option, and the providing the candidate meme in response to the selection of the selectable option.

3. The method of claim 1 , wherein the determining whether the meme suggestion mode should be activated based on the electronic communication includes determining whether a selectable control for enabling the meme suggestion mode is selected.

4. The method of claim 3 , wherein the selectable control for enabling the meme suggestion mode includes generating for display a selectable switch or a selectable button in the electronic communication.

5. The method of claim 3 , wherein the selectable control for enabling the meme suggestion mode includes an always on mode, an always off mode, and a smart decision mode.

6. The method of claim 1 , comprising:

in response to determining the context of the portion of the electronic communication is informal, activating the meme suggestion mode.

7. The method of claim 1 , wherein the textual response is further based on at least one of an image-to-text process, a user profile, or a meme template.

8. The method of claim 1 , comprising:

in response to determining the meme suggestion mode should be activated:

determining whether the electronic communication includes a question; and

in response to determining the electronic communication includes the question:

determining an answer to the question; and

including the answer in the textual response.

9. A system comprising:

circuitry configured to:

access a portion of an electronic communication, wherein the accessing the portion of the electronic communication includes determining a context of the portion of the electronic communication;

determine a textual response and a visual response to the portion of the electronic communication;

determine whether a meme suggestion mode should be activated based on the electronic communication;

generate a candidate meme comprising an image including the textual response and the visual response;

provide for display as part of the electronic communication a selectable option including display of the candidate meme; and

in response to selection of the selectable option, provide the candidate meme for display within the electronic communication, wherein the textual response is based at least in part on the context determined from the portion of the electronic communication.

10. The system of claim 9 , wherein the circuitry is configured to:

in response to determining the meme suggestion mode should be activated, execute the determining, the generating, the providing of the selectable option, and the providing the candidate meme in response to the selection of the selectable option.

11. The system of claim 9 , wherein the circuitry configured to determine whether the meme suggestion mode should be activated based on the electronic communication is configured to determine whether a selectable control for enabling the meme suggestion mode is selected.

12. The system of claim 11 , wherein the circuitry is configured to:

generate for display the selectable control for enabling the meme suggestion mode, and

generate for display a selectable switch or a selectable button in the electronic communication.

13. The system of claim 11 , wherein the circuitry is configured to:

generate for display the selectable control including an always on mode, an always off mode, and a smart decision mode.

14. The system of claim 9 , wherein the circuitry is configured to:

in response to determining the context of the portion of the electronic communication is informal, activate the meme suggestion mode.

15. The system of claim 9 , wherein the circuitry is further configured to:

generate the textual response based on at least one of an image-to-text process, a user profile, or a meme template.

16. A method for enhanced content generation during an electronic conversation to generate a meme, the meme includes at least a candidate image and a candidate caption, comprising:

accessing a portion of the electronic communication;

determining whether the meme suggestion mode should be activated based on the electronic communication;

generating the candidate image based on at least one of on at least one of a context determined from a portion of the electronic communication, a text-to-image process, a user profile, a user-provided image, or an enhanced content template;

generating the candidate caption based on at least one of the context determined from the portion of the electronic communication, an image-to-text process, the user profile, or the enhanced content template;

determining all possible combinations of the candidate image and the candidate caption;

analyzing the all possible combinations for a likelihood that a candidate enhanced content will be selected by a user;

generating for display a number of top rated candidate enhanced contents;

providing a user selectable option for selecting one of the top rated candidate enhanced contents; and

in response to user selection of the selected one of the top rated candidate enhanced contents, generating for display within the electronic communication the selected one of the top rated candidate enhanced contents.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2023
From: XU, NING; DOKEN, SERHAD; HARB, REDA
To: ADEIA GUIDES INC.
Reel/Frame 064693/0932 →
Continuity (1)
Related Publication 20250014245A1 · Jan 9, 2025
References Cited (45)
US 10015124B2 · McGregor, Jr. · 2018 [cited by examiner]
US 10567312B2 · Goldberg · 2020 [cited by examiner]
US 10853717B2 · Abramson · 2020 [cited by examiner]
US 11050694B2 · Rodriguez · 2021 [cited by examiner]
US 11093711B2 · Iida · 2021 [cited by examiner]
US 11232406B1 · Michelson · 2022 [cited by examiner]
US 11825004B1 · Halstead · 2023 [cited by examiner]
US 12260374B2 · Michelson · 2025 [cited by examiner]
US 20060015560A1 · MacAuley · 2006 [cited by examiner]
US 20060089147A1 · Beaty · 2006 [cited by examiner]
US 20120270578A1 · Feghali · 2012 [cited by examiner]
US 20150271119A1 · Goncalves · 2015 [cited by examiner]
US 20160080298A1 · Oh · 2016 [cited by examiner]
US 20180314409A1 · Adilipour · 2018 [cited by examiner]
US 20200007958A1 · Wondra · 2020 [cited by examiner]
US 20200382454A1 · Gershony · 2020 [cited by examiner]
US 20210158594A1 · Huang · 2021 [cited by examiner]
US 20210314284A1 · Zhai · 2021 [cited by examiner]
US 20220147206A1 · Zhai · 2022 [cited by examiner]
US 20220182343A1 · Lee · 2022 [cited by examiner]
US 20220360551A1 · Cha · 2022 [cited by examiner]
US 20230066331A1 · Kumar · 2023 [cited by examiner]
US 20230105825A1 · Agarwal · 2023 [cited by examiner]
US 20230300095A1 · Chen · 2023 [cited by examiner]
US 20230333729A1 · Lin · 2023 [cited by examiner]
US 20230333867A1 · Agarwal · 2023 [cited by examiner]
US 20240143149A1 · Duperron · 2024 [cited by examiner]
US 20240195770A1 · Cheng · 2024 [cited by examiner]
US 20250014245A1 · Xu · 2025 [cited by examiner]
Amra & Elma, “Top Meme Statistics in 2022.” Dec. 1, 2022. Accessed via The Wayback Machine on May 26, 2023. [cited by applicant]
BroadbandSearch, “Average Time Spent Daily on Social Media.” Nov. 1, 2019. Accessed via The Wayback Machine on May 26, 2023. [cited by applicant]
Canva, “Introvert Koala Simple Photo and Text Meme.” Nov. 17, 2022. Accessed via The Wayback Machine on May 26, 2023. [cited by applicant]
Chudasama, Vishal, et al. “M2FNet: multi-modal fusion network for emotion recognition in conversation.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. [cited by applicant]
Devlin, Jacob, et al. “Bert: Pre-training of deep bidirectional transformers for language understanding.” arXiv preprint arXiv:1810.04805 (2019). [cited by applicant]
Digital Information World, “Over 1 in Every 5 Tweets Contain an Emoji, New Study Reveals.” Jul. 25, 2021. Accessed via The Wayback Machine on May 26, 2023. [cited by applicant]
Greyling, C. “Using GPT-3 to Measure Sentiment.” Medium. Jun. 13, 2021. Accessed via The Wayback Machine on May 26, 2023. [cited by applicant]
Imgflip, “This Meme Does Not Exist.” Apr. 11, 2019. Accessed via The Wayback Machine on May 26, 2023. [cited by applicant]
Liu, Qian, et al. “You impress me: Dialogue generation via mutual persona perception.” arXiv preprint arXiv:2004.05388 (2020). [cited by applicant]
Liu, Zhiyuan, et al. “Multi-modal application: Image Memes Generation.” arXiv preprint arXiv:2112.01651 (2021). [cited by applicant]
OpenAI, “Customizing GPT-3 for Your Application.” Dec. 14, 2021. Accessed via The Wayback Machine on May 26, 2023. [cited by applicant]
Peirson V, Abel L., and E. Meltem Tolunay. “Dank learning: Generating memes using deep neural networks,” arXiv preprint arXiv:1806.04510 (2018). [cited by applicant]
Ruiz, Nataniel, et al. “Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023. [cited by applicant]
Sadasivam, Aadhavan. memeBot: Automatic Image Meme Generation for Online Social Interaction. Diss. Arizona State University, 2020. [cited by applicant]
Song, Xiaohui, et al. “Emotionflow: Capture the dialogue level emotion transitions.” ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022. [cited by applicant]
Sung, M. “The AI meme generator is better at making memes than humans.” Mashable. Apr. 29, 2020. Accessed via The Wayback Machine on May 26, 2023. [cited by applicant]