IP Library Granted Patent US 10,129,198
Granted Patent B2
US 10,129,198 · App. 15/793,683 · Granted Nov 13, 2018

Contextually driven messaging system

Inventor: Evan Nicklas Wu Malahy (Sunnyvale, CA)
Assignee: Google LLC
H04L51/24G06Q10/107G10L13/02H04L51/00H04M1/72552G10L15/26H04L51/02H04M1/72569
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,129,198
App. No.
15/793,683
Granted
Nov 13, 2018
Kind
B2
Abstract

A method may include receiving, by a computing device associated with a user, a message from an origination source and receiving, by the computing device, an audio input. The method may also include determining, by the computing device and based at least in part on the audio input and contextual information, a probability that the user intends to send a response message to the origination source. The method may further include, responsive to determining that the probability the user intends to send the response message to the origination source satisfies a threshold probability, determining, by the computing device, that the user intends to send the response message to the origination source. The method may also include, responsive to determining that the user intends to send the response message to the origination source, generating, by the computing device and based on the audio input, the response message, and sending, by the computing device, the response message to the origination source.

Claims (100)

1. A method for automatically detecting a conversation and determining an intended recipient of an outgoing communication, the method comprising:

receiving, by a computing device associated with a first user, a message from a device associated with a second user;

receiving, by the computing device, an audio input;

determining, by the computing device and without outputting a request for instructions to the first user, and based at least in part on the message from the device associated with the second user, the audio input, and contextual information, whether the first user and the second user are engaged in the conversation;

responsive to determining that the first user and the second user are engaged in the conversation:

determining, by the computing device, that the second user is the intended recipient of the outgoing communication;

generating, by the computing device and based on the audio input, the outgoing communication as a response to the message received from the device associated with the second user; and

sending, by the computing device and without receiving user confirmation of the contents of the outgoing communication or a user instruction to send the outgoing communication, the outgoing communication to the second user.

2. The method of claim 1 , wherein determining that the first user and the second user are engaged in the conversation comprises:

determining, by the computing device and based at least in part on the message from the device associated with the second user, the audio input, and the contextual information, a probability that the first user intends to send the response to the message received from the device associated with the second user,

wherein determining whether the first user and the second user are engaged in the conversation is based at least in part on the probability.

3. The method of claim 2 , further comprising:

prior to receiving the audio input, receiving, by the computing device, a message from a device associated with a third user;

after receiving the message from the device associated with the second user:

determining, by the computing device and based at least in part on the audio input, and the contextual information, a probability that the first user intends to send the outgoing communication as a response to the message received from the device associated with the third user;

determining, by the computing device and based on the probability that the first user intends to send the outgoing communication as the response to the message received from the device associated with the second user and the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user, whether the user intends to send the outgoing communication as the response to the message received from the device associated with the second user or to the message received from the device associated with the third user; and

responsive to determining that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user:

generating, by the computing device and based on the audio input, the response to the message received from the device associated with the third user; and

sending, by the computing device, the response to the message received from the device associated with the third user.

4. The method of claim 3 ,

wherein determining that the user intends to send the outgoing communication as the response to the message received from the device associated with the second user is further responsive to determining that the probability the user intends to send the outgoing communication as the response to the message received from the device associated with the second user satisfies the threshold probability but the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user does not satisfy the threshold probability, and

wherein determining that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user is further responsive to determining that the probability the user intends to send the outgoing communication as the response to the message received from the device associated with the third user satisfies the threshold probability but the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the second user does not satisfy than the threshold probability.

5. The method of claim 3 ,

wherein determining that the user intends to send the outgoing communication as the response to the message received from the device associated with the second user is further responsive to determining that the probability the user intends to send the outgoing communication as the response to the message received from the device associated with the second user satisfies the threshold probability and is greater than the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user, and

wherein determining that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user is further responsive to determining that the probability the user intends to send the outgoing communication as the response to the message received from the device associated with the third user satisfies the threshold probability and is greater than the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the second user.

6. The method of claim 1 , further comprising:

determining, by the computing device, a probability the user intends to listen to the message received from the device associated with the second user; and

responsive to determining that the probability the user intends to listen to the message received from the device associated with the second user satisfies a threshold probability for listening to messages:

generating, by the computing device and based on the message received from the device associated with the second user, audio data; and

outputting, by the computing device, the audio data.

7. The method of claim 1 , wherein the contextual information includes one or more of:

a frequency of incoming messages from the device associated with the second user, a frequency of outgoing messages to the device associated with the second user, an amount of time that has elapsed since a last received message from the device associated with the second user, or an amount of time that has elapsed since a last sent message to the device associated with the second user.

8. The method of claim 2 , wherein the probability that the user intends to send the response to the message to the device associated with the second user is not based a user command.

9. A device associated with a first user, the device comprising:

at least one processor;

an audio input device;

a communication unit; and

a storage device that stores a message management module, wherein the at least one processor executes the message management module to:

receive, via the communication unit, a message from a device associated with a second user;

receive, via the audio input device, an audio input;

determine, without outputting a request for instructions to the first user and based at least in part on the message from the device associated with the second user, the audio input, and contextual information, whether the first user and the second user are engaged in a conversation; and

responsive to determining that the first user and the second user are engaged in the conversation:

determine that the second user is an intended recipient of the outgoing communication;

generate, based on the audio input, an outgoing communication as a response to the message received from the device associated with the second user; and

send, via the communication unit, and without receiving user confirmation of the contents of the response or a user instruction to send the response, the outgoing communication to the second user.

10. The device of claim 9 , wherein the at least one processor further executes the message management module to:

determine, based at least in part on the message from the device associated with the second user, the audio input, and the contextual information, a probability that the first user intends to send the response to the message received from the device associated with the second user; and

determine whether the first user and the second user are engaged in the conversation based at least in part on the probability.

11. The device of claim 10 , further comprising:

an audio output device,

wherein the at least one processor further executes the message management module to:

responsive to determining that the probability the user intends to send the response to the message received from the device associated with the second user does not satisfy the threshold probability:

output, via the audio output device, a request for additional action by the first user;

receive, via the audio input device; a second audio input indicative of the user's intent to send the response to the message received from the device associated with the second user; and

send, via the communication unit and based at least in part on the second audio input, the response to the message.

12. The device of claim 10 , wherein the at least one processor further executes the message management module to:

prior to receiving the audio input, receive, via the communication unit, a message from a device associated with a third user;

after receiving the message from the device associated with the second user:

determine, based at least in part on the audio input and contextual information, a probability that the first user intends to send the outgoing communication as a response to the message received from the device associated with the third user;

determine, based on the probability that the first user intends to send the outgoing communication as the response to the message received from the device associated with the second user and the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user, whether the user intends to send the outgoing communication as the response to the message received from the device associated with the second user or to the message received from the device associated with the third user; and

responsive to determining that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user:

generate, based on the audio input, the outgoing communication as the response to the message received from the device associated with the third user; and

send, via the communication network, the outgoing communication as the response to the message received from the device associated with the third user.

13. The device of claim 12 ,

wherein the at least one processors executes the message management module to determine that the user intends to send the outgoing communication as the response to the message received from the device associated with the second user responsive to determining that the probability the user intends to send the outgoing communication as the response to the message received from the device associated with the second user satisfies the threshold probability but the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user does not satisfy the threshold probability, and

wherein the at least one processors executes the message management module to determine that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user is further responsive to determining that the probability the user intends to send the outgoing communication as the response to the message received from the device associated with the third user satisfies the threshold probability but the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the second user does not satisfy than the threshold probability.

14. The device of claim 12 ,

wherein the at least one processors executes the message management module to determine that the user intends to send the outgoing communication as the response to the message received from the device associated with the second user is further responsive to determining that the probability the user intends to send the outgoing communication as the response to the message received from the device associated with the second user satisfies the threshold probability and is greater than the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user, and

wherein the at least one processors executes the message management module to determine that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user is further responsive to determining that the probability the user intends to send the outgoing communication as the response to the message received from the device associated with the third user satisfies the threshold probability and is greater than the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the second user.

15. The device of claim 9 , further comprising:

an audio output device,

wherein the at least one processor further executes the message management module to:

determine a probability the user intends to listen to the message received from the device associated with the second user; and

responsive to determining that the probability the user intends to listen to the message received from the device associated with the second user satisfies a threshold probability for listening to messages:

generate, based on the message received from the device associated with the second user, audio data; and

output, via the audio output device, the audio data.

16. The device of claim 9 , wherein the contextual information includes one or more of:

frequency of incoming messages from the device associated with the second user, frequency of outgoing messages to the device associated with the second user, time since the last received message from the device associated with the second user, or time since the last sent message to the device associated with the second user.

17. The device of claim 9 , wherein the probability that the user intends to send the response to the message received from the device associated with the second user is not based a user command.

18. A non-transitory computer-readable storage medium comprising instructions that, when executed, configure one or more processors of a computing system to:

receive a message from a device associated with a second user;

receive an audio input;

determine, without outputting a request for instructions to the first user and based at least in part on the message from the device associated with the second user, the audio input, and contextual information, whether the first user and the second user are engaged in a conversation;

responsive to determining that the first user and the second user are engaged in the conversation:

determine that the second user is an intended recipient of the outgoing communication;

generate, based on the audio input, an outgoing communication as a response to the message received from the device associated with the second user; and

send, without receiving user confirmation of the contents of the response or a user instruction to send the response, the response to the message to the second user.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions, when executed, further configure the one or more processors to:

prior to receiving the audio input, receive, via the communication unit, a message from a device associated with a third user;

after receiving the message from the device associated with the second user:

determine, based at least in part on the audio input and the contextual information, a probability that the first user intends to send the outgoing communication as a response to the message received from the device associated with the third user;

determine, based on the probability that the first user intends to send the outgoing communication as the response to the message received from the device associated with the second user and the probability that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user, whether the user intends to send the outgoing communication as the response to the message received from the device associated with the second user or to the message received from the device associated with the third user; and

responsive to determining that the user intends to send the outgoing communication as the response to the message received from the device associated with the third user:

generate, based on the audio input, the outgoing communication as the response to the message received from the device associated with the third user; and

send the outgoing communication as the response to the message received from the device associated with the third user.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions, when executed, further configure the one or more processors to:

determine a probability the user intends to listen to the message received from the device associated with the second user; and

responsive to determining that the probability the user intends to listen to the message received from the device associated with the second user satisfies a threshold probability for listening to messages:

generate, based on the message received from the device associated with the second user, audio data; and

output the audio data.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2017
From: MALAHY, EVAN NICKLAS WU
To: GOOGLE INC.
Reel/Frame 044363/0491 →
CHANGE OF NAME Recorded Dec 12, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044836/0708 →
Continuity (2)
Continuation 14736226 · Jun 10, 2015
Related Publication 20180048610A1 · Feb 15, 2018