Personality reply for digital content

Inventors: Ibrahim Badr (Zurich, CH); Aayush Kumar (Zurich, CH); Goekhan Hasan Bakir (Zurich, CH); Nils Grimsmo (Adliswil, CH); Bianca Madalina Buisman (Kilchberg, CH)

Assignee: GOOGLE LLC

H04L51/046 H04L51/10

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,405,340
App. No.: 17/019,575
Granted: Aug 2, 2022
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

A computer-implemented method is described. The method includes a computing system receiving an item of digital content from a user device. The computing system generates one or more labels that indicate attributes of the item of digital content. The computing system also generates one or more conversational replies to the item of digital content based on the one or more labels that indicate attributes of the item of digital content. The method also includes the computing system selecting a conversational reply from among the one or more conversational replies and providing the conversational reply for output to the user device.

Claims (56)

1. A method implemented by one or more processors comprising:

receiving an electronic communication that includes an image, the electronic communication being received at a user device of a user and from an additional user device of an additional user;

processing the image to identify one or more items included in the image;

generating, based on one or more of the items included in the image, multiple candidate replies that are each descriptive of the item in the image,

wherein generating the multiple candidate replies comprises generating a given candidate reply, of the multiple candidate replies, based on predetermined replies stored in one or more of the databases that are related to one or more of the items included in the image, and

wherein generating the given candidate reply based on the predetermined replies stored in one or more of the databases that are related to one or more of the items included in the image comprises:

identifying a predetermined conversational reply, from one or more of the databases, that is associated with one or more of the items included in the image; and

generating the given candidate reply based on the predetermined conversational reply that is associated with one or more of the items included in the image;

providing, for output as a suggested reply to the electronic communication at the user device of the user, one or more of the multiple candidate replies;

receiving, from the user of the user device, a selection of one of the one or more multiple candidate replies; and

causing the selected one of the one or more multiple candidate replies to be transmitted to the additional user device of the additional user.

2. The method of claim 1 , wherein processing the image to identify one or more of the items included in the image comprises:

generating a label for one or more of the items included in the image,

wherein identifying the predetermined conversational reply is based on the label for one or more of the items included in the image.

3. The method of claim 1 , wherein one or more of the databases include a plurality of curated strings of text or words that are descriptive of one or more of the items included in the image.

4. The method of claim 3 , wherein the plurality of curated strings of text or words are defined by a human.

5. The method of claim 1 , wherein providing one or more of the multiple candidate replies for output as the suggested reply to the electronic communication at the user device of the user comprises:

visually rendering, via an electronic communication interface of the user device, the suggested reply along with the electronic communication received at the user device of the user.

6. A system comprising:

at least one processor; and

at least one memory storing instructions that, when executed, cause the at least one processor to:

receive an electronic communication that includes an image, the electronic communication being received at a user device of a user and from an additional user device of an additional user;

process the image to identify one or more items included in the image;

generate, based on one or more of the items included in the image, multiple candidate replies that are each descriptive of the item in the image,

wherein the instructions to generate the given candidate reply based on the predetermined replies stored in one or more of the databases that are related to one or more of the items included in the image comprise instructions to:

identify a predetermined conversational reply, from one or more of the databases, that is associated with one or more of the items included in the image; and

generate the given candidate reply based on the predetermined conversational reply that is associated with one or more of the items included in the image;

provide, for output as a suggested reply to the electronic communication at the user device of the user, one or more of the multiple candidate replies;

receive, from the user of the user device, a selection of one of the one or more multiple candidate replies; and

cause the selected one of the one or more multiple candidate replies to be transmitted to the additional user device of the additional user.

7. The system of claim 6 , wherein the instructions to process the image to identify one or more of the items included in the image comprise instructions to:

generate a label for one or more of the items included in the image,

wherein identifying the predetermined conversational reply is based on the label for one or more of the items included in the image.

8. The system of claim 6 , wherein one or more of the databases include a plurality of curated strings of text or words that are descriptive of one or more of the items included in the image.

9. The system of claim 8 , wherein the plurality of curated strings of text or words are defined by a human.

10. The system of claim 6 , wherein the instructions to provide one or more of the multiple candidate replies for output as the suggested reply to the electronic communication at the user device of the user comprise instructions to:

visually render, via an electronic communication interface of the user device, the suggested reply along with the electronic communication received at the user device of the user.

11. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor to perform operations, the operations comprising: