IP Library Granted Patent US 9,325,809
Granted Patent B1
US 9,325,809 · App. 13/973,922 · Granted Apr 26, 2016

Audio recall during voice conversations

Inventors: Brett Barros (San Francisco, CA); Moninder Jheeta (Redmond, WA); George Samuel Kola (Concord, CA); Timothy Tuttle (San Francisco, CA)
Assignee: Mindmeld, Inc.
H04L67/42
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,325,809
App. No.
13/973,922
Granted
Apr 26, 2016
Kind
B1
Abstract

Conversation information is provided to a conversation participant. Audio data is received a client device associated with a conversation participant. The client device maintains a buffer of received audio data, for instance audio data received in a previous interval of time. A recall request is received by the client device for information associated with the buffered audio data. The buffered audio data is analyzed responsive to the recall request to identify the requested information. The identified information is then presented at the client device.

Claims (54)

1. A computer-implemented method for providing conversation information to participants of a conversation, comprising:

buffering, at a client device, audio data received from one or more conversation participants;

receiving, at the client device, a recall request for information associated with the buffered audio data from a conversation participant;

without pausing the conversation:

notifying client devices associated with other conversation participants of the recall request, each client device associated with another conversation participant configured to buffer conversation data received from one or more conversation participants;

receiving, from the notified client devices, information representative of buffered conversation data;

analyzing, by the client device, the buffered audio data and the received information representative of buffered conversation data to identify one or more topics of the conversation associated with the analyzed data;

identifying content associated with the identified one or more topics; and

presenting, at the client device in real time, the identified content.

2. The computer-implemented method of claim 1 , further comprising:

capturing, by the client device, audio data from a user of the client device participating in the conversation.

3. The computer-implemented method of claim 1 , wherein buffering audio data comprises storing the received audio data in conjunction with time data representing a time at which the audio data was received.

4. The computer-implemented method of claim 1 , wherein analyzing the buffered audio data and the received information representative of buffered conversation data comprises:

determining one or more conversation parameters associated with the conversation.

5. The computer-implemented method of claim 4 , wherein determining one or more conversation parameters associated with the conversation:

converting the buffered audio data into a text transcript;

identifying keywords within the text transcript relevant to the conversation; and

identifying one or more topics of the conversation based on the identified keywords.

6. The computer-implemented method of claim 5 , wherein the identified keywords comprise one or more of: nouns, noun phrases, verbs, and verb phrases that appear within the text transcript over a threshold frequency or a threshold number of instances; terms predetermined to be relevant; terms dynamically determined to be relevant; and terms associated with brands, products, services, or other commercial interests.

7. The computer-implemented method of claim 5 , wherein identifying one or more topics of the conversation based on the identified keywords comprises:

identifying a topic keyword based on the identified keywords, the topic keyword not included in the identified keywords, wherein an identified topic of conversation comprises the identified topic keyword.

8. The computer-implemented method of claim 4 , wherein determining one or more conversation parameters associated with the conversation comprises:

accessing one or more of the buffered audio data and characteristics or information associated with conversation participants; and

determining one or more of relationships between conversation participants, measures of affinity between conversation participants, and a context for conversation participants.

9. The computer-implemented method of claim 4 , wherein the determined one or more conversation parameters comprise one or more of: concepts, names, places, entities, and phrases.

10. The computer-implemented method of claim 1 , wherein the client device and the one or more notified client devices are configured to provide text transcripts of audio data buffered by each client device to a central server, and further comprising:

receiving, from the central server, one or more conversation parameters, the central server configured to analyze the text transcripts provided by the client device and the notified client devices to identify the one or more conversation parameters.

11. The computer-implemented method of claim 1 , wherein presenting the identified content comprises displaying the identified content on a client device display.

12. The computer-implemented method of claim 11 , wherein the displayed content includes one or more of: a time that a portion of audio data associated with the requested information was spoken by a conversation participant or captured by the client device, an identity of a conversation participant associated with the requested information, and information derived from the identified conversation parameters.

13. The computer-implemented method of claim 11 , wherein presenting the identified content comprises presenting an interface configured to allow a user of the client device to perform a search based one or more identified conversation parameters.

14. A non-transitory computer-readable storage medium storing executable computer program instructions for providing conversation information to participants of a conversation, the instructions performing steps comprising:

buffering, at a client device, audio data received from one or more conversation participants;

receiving, at the client device, a recall request for information associated with the buffered audio data from a conversation participant;

without pausing the conversation:

notifying client devices associated with other conversation participants of the recall request, each client device associated with another conversation participant configured to buffer conversation data received from one or more conversation participants;

receiving, from the notified client devices, information representative of buffered conversation data;

analyzing, by the client device, the buffered audio data and the received information representative of buffered conversation data to identify one or more topics of the conversation associated with the analyzed data;

identifying content associated with the identified one or more topics; and

presenting, at the client device in real time, the identified content.

15. The computer-readable storage medium of claim 14 , wherein the client device and the one or more notified client devices are configured to provide text transcripts of audio data buffered by each client device to a central server, and the instructions performing further steps comprising:

receiving, from the central server, one or more conversation parameters, the central server configured to analyze the text transcripts provided by the client device and the notified client devices to identify the one or more conversation parameters.

16. A computer system for providing conversation information to participants of a conversation, comprising:

a non-transitory computer readable storage medium storing executable computer program instructions comprising instructions for:

buffering, at the computer system, audio data received from one or more conversation participants;

receiving, at the computer system, a recall request for information associated with the buffered audio data from a conversation participant;

without pausing the conversation:

notifying client devices associated with other conversation participants of the recall request, each client device associated with other conversation participants configured to buffer conversation data received from one or more conversation participants;

receiving, from the notified client devices, information representative of buffered conversation data;

analyzing, by the client device, the buffered audio data and the received information representative of buffered conversation data to identify one or more topics of the conversation associated with the analyzed data;

identifying content associated with the identified one or more topics; and

presenting, at the client device in real time, the identified content; and

a processor for executing the computer program instructions.

17. The computer system of claim 16 , wherein the computer system and the one or more notified client devices are configured to provide text transcripts of audio data buffered by the computer system and each notified client device to a central server, the instructions further comprising instructions for:

receiving, from the central server, one or more conversation parameters, the central server configured to analyze the text transcripts provided by the computer system and the notified client devices to identify the one or more conversation parameters.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 14, 2017
From: CISCO MINDMELD LLC
To: CISCO TECHNOLOGY, INC.
Reel/Frame 044402/0323 →
CHANGE OF NAME Recorded Dec 14, 2017
From: MINDMELD, INC.
To: CISCO MINDMELD LLC
Reel/Frame 044883/0850 →
CHANGE OF NAME Recorded Dec 21, 2015
From: EXPECT LABS, INC.
To: MINDMELD, INC.
Reel/Frame 037354/0930 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 26, 2013
From: BARROS, BRETT; JHEETA, MONINDER; KOLA, GEORGE SAMUEL; TUTTLE, TIMOTHY
To: EXPECT LABS, INC.
Reel/Frame 031083/0745 →
Continuity (1)
Provisional Application 61698333 · Sep 7, 2012