IP Library Granted Patent US 11,069,103
Granted Patent B1
US 11,069,103 · App. 15/492,077 · Granted Jul 20, 2021

Customized user interface for electronic communications

Inventors: Jacob Edward Blackstock (Toronto, CA); Diego Muñoz Escalante (Toronto, CA); Matthew Colin Grantham (Toronto, CA); William Wu (Venice, CA)
Assignee: Snap Inc.
G06T11/60G06K9/00228
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,069,103
App. No.
15/492,077
Granted
Jul 20, 2021
Kind
B1
Abstract

Among other things, embodiments of the present disclosure improve the functionality of electronic messaging software and systems by providing a customized display of avatars corresponding to users exchanging electronic communications, such as short message service (SMS) or multimedia service (MMS) texts and emails. The appearance of the displayed avatars may be dynamically altered based on the actions taken by the users, as well as based on the content of the messages between the users or information from other sources.

Claims (66)

1. A system comprising:

a processor; and

memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising:

receiving, over the Internet, an electronic communication from a first computing device of a first user;

transmitting, over the Internet, the electronic communication to a second computing device of a second user;

analyzing content within the electronic communication to identify a tone associated with the content;

determining, based on the identified tone, a mood of the first user;

retrieving, from a database in communication with system, avatar characteristics associated with the first user;

generating a media overlay which includes an avatar of the first user, the avatar of the first user being based on the retrieved avatar characteristics associated with the first user, the media overlay including additional image or text content that is based on the determined mood of the first user;

generating a media content item which includes the media overlay combined with an image feed captured by a camera coupled to the first computing device;

causing presentation of the electronic communication and the media content item within a chat interface window displayed on a respective display screen coupled to the first computing device and the second computing device,

wherein the chat interface window corresponds to a group conversation between participants including the first user and the second user, and

wherein the chat interface window includes a border along which respective avatars of the participants are displayed.

2. The system of claim 1 , wherein the memory further stores instructions for causing the system to perform operations comprising:

analyzing content within a plurality of electronic communications exchanged between the first computing device and the second computing device to identify a tone associated with the content;

determining the mood of the first user and a mood of the second user based on the identified tone of the content;

retrieving, from the database coupled to the system, avatar characteristics associated with the second user; and

generating an avatar of the second user based on the retrieved avatar characteristics associated with the second user and the determined mood of the second user.

3. The system of claim 1 , wherein generating the avatar of the first user includes:

retrieving an image of the first user's face via the camera coupled to the first computing device; and

generating facial features of the avatar based on the image of the first user's face.

4. The system of claim 1 , wherein the content within the electronic communication includes audio content, and wherein determining the mood of the first user includes performing a voice analysis on the audio content.

5. The system of claim 1 , wherein the content within the electronic communication includes text input, and wherein determining the mood of the first user includes performing a text analysis on the text input.

6. The system of claim 1 , wherein determining the mood of the first user is based on input from the first user via the first computing device, the input comprising one or more mood selections.

7. The system of claim 1 , further comprising:

receiving supplemental information specific to the first user, wherein determining the mood of the first user is further based on the supplemental information, and

wherein the supplemental information corresponds to at least one of:

a calendar entry stored on the first computing device, the calendar entry being specific to the first user,

a social network entry stored in association with a user account corresponding to the first user, or

motion sensor data which is associated with the first computing device and which corresponds to a user activity of the first user.

8. The system of claim 1 , wherein determining the mood of the first user includes determining a level of the mood among a plurality of mood levels, the level of the mood comprising a numeric scale, and

wherein the plurality of mood levels includes a first mood level and a second mood level, the first mood level having a lesser mood level than the second mood level.

9. The system of claim 8 , wherein the system determines the mood level for the first user as the first mood level, and wherein generating the avatar of the first user includes depicting a first expression on a face of the avatar corresponding to the first mood level.

10. The system of claim 9 , wherein the system determines the mood level for the first user as the second mood level, and wherein generating the avatar of the first user includes depicting a second expression on the face of the avatar corresponding to the second mood level, the second expression being different from the first expression.

11. The system of claim 1 , wherein determining the mood for the first user is further based on a relationship between the first user and the second user.

12. The system of claim 11 , wherein the group conversation is between the first user, the second user, and a third user, wherein determining the mood for the first user includes determining a second mood for the first user based on the identified tone and a relationship between the first user and the third user, and wherein generating the avatar of the first user includes:

generating a first avatar for the first user based on the retrieved avatar characteristics and the determined mood of the first user; and

generating a second avatar for the first user based on the retrieved avatar characteristics and the second determined mood of the first user.

13. The system of claim 12 , wherein causing presentation of the avatar of the first user includes:

causing presentation of the first avatar of the first user on the display screen of the second computing device; and

causing presentation of the second avatar of the first user on a display screen of a third computing device of the third user.

14. The system of claim 1 , further comprising:

determining that the first user has muted the group conversation; and

causing, based on the first user muting the group conversation, presentation of the avatar of the first user to be in a contracted state with respect to the border of the chat interface window.

15. A computer-implemented method comprising:

receiving, by a computer system over the Internet, an electronic communication from a first computing device of a first user;

transmitting, by the computer system over the Internet, the electronic communication to a second computing device of a second user;

analyzing, by the computer system, content within the electronic communication to identify a tone associated with the content;

determining, by the computer system based on the identified tone, a mood of the first user;

retrieving, by the computer system from a database in communication with computer system, avatar characteristics associated with the first user;

generating, by the computer system, a media overlay which includes an avatar of the first user, the avatar of the first user being based on the retrieved avatar characteristics associated with the first user, the media overlay including additional image or text content that is based on the determined mood of the first user;

generating, by the computer system, a media content item which includes the media overlay combined with an image feed captured by a camera coupled to the first computing device;

causing, by the computer system, presentation of the electronic communication and the media content item within a chat interface window displayed on a respective display screen coupled to the first computing device and the second computing device,

wherein the chat interface window corresponds to a group conversation between participants including the first user and the second user, and

wherein the chat interface window includes a border along which respective avatars of the participants are displayed.

16. A non-transitory computer-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations comprising:

receiving, over the Internet, an electronic communication from a first computing device of a first user;

transmitting, over the Internet, the electronic communication to a second computing device of a second user;

analyzing content within the electronic communication to identify a tone associated with the content;

determining, based on the identified tone, a mood of the first user;

retrieving, from a database in communication with computer system, avatar characteristics associated with the first user;

generating a media overlay which includes an avatar of the first user, the avatar of the first user being based on the retrieved avatar characteristics associated with the first user, the media overlay including additional image or text content that is based on the determined mood of the first user;

generating a media content item which includes the media overlay combined with an image feed captured by a camera coupled to the first computing device;

causing presentation of the electronic communication and the media content item within a chat interface window displayed on a respective display screen coupled to the first computing device and the second computing device,

wherein the chat interface window corresponds to a group conversation between participants including the first user and the second user, and

wherein the chat interface window includes a border along which respective avatars of the participants are displayed.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 17, 2021
From: BLACKSTOCK, JACOB EDWARD; ESCALANTE, DIEGO MUÑOZ; GRANTHAM, MATTHEW COLIN; WU, WILLIAM
To: SNAP INC.
Reel/Frame 056573/0356 →
Cited By (70)
US 1,089,291 US 12,198,398 US 12,223,672 US 12,229,901 US 12,235,991 US 12,236,512 US 12,254,577 US 12,271,536 US 12,277,632 US 12,284,146 US 12,284,698 US 12,288,273 US 12,293,433 US 12,299,775 US 12,307,564 US 12,315,495 US 12,321,577 US 12,340,453 US 12,361,934 US 12,387,444 US 12,394,154 US 12,395,456 US 12,412,347 US 12,417,007 US 12,417,562 US 12,418,504 US 12,429,953 US 12,436,598 US 12,464,064 US 12,469,273 US 12,472,435 US 12,475,621 US 12,475,658 US 12,499,483 US 12,499,638 US 12,504,866 US 12,513,098 US 12,517,626 US 12,518,437 US 12,518,738 US 12,530,847 US 12,530,852 US 12,536,751 US 12,541,930 US 12,548,267 US 12,555,274 US 12,555,310 US 12,567,102 US 12,579,204 US 12,580,784 US 12,580,980 US 12,586,562 US 12,602,842 US 12,614,354 US 12,614,359 US 12,620,188 US 12,620,216 US 12,632,890 US 12,633,073 US 12,646,266 US 12,646,268 US 12,651,292 US 12,651,409 US 12,670,674 US 12,682,508 US 12,695,929 US 12,699,802 US 12,700,187 US 12,705,875 US 12,711,538