IP Library Granted Patent US 11,093,718
Granted Patent B1
US 11,093,718 · App. 17/246,463 · Granted Aug 17, 2021

Determining conversational structure from speech

Inventors: Toshish Arun Jawale (Seattle, WA); Ansup Babu (Balangir, IN); Anthony Claudia (Boulder, CO)
Assignee: Rammer Technologies, Inc.
G06F40/35G06F40/289
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,093,718
App. No.
17/246,463
Granted
Aug 17, 2021
Kind
B1
Abstract

Embodiments are directed to organizing conversations. Words may be provided from a conversation stream. Each word may be mapped to a graph model based on characteristics of each word. The graph model may be partitioned based on one or more attributes of a nodes and edges included in the graph model such that nodes associated with relationship strength that exceeds a threshold value may be grouped together. Sentence models may be generated based on sentences included in the conversation stream. Combined models may be generated based on the sentence models and the graph such that each sentence model may be associated with one or more partitions of the graph model. A conversation digest may be generated based on the combined model such that the conversation digest identifies one or more dominant portions of the conversation that include key subject matter.

Claims (109)

1. A method for organizing conversations over a network using one or more network computers that include one or more processors that perform actions, comprising:

providing a graph model for a conversation based on one or more words provided by a conversation stream, wherein each word is associated with a node in the graph model and each edge in the graph model corresponds to one or more relationships between each word;

generating one or more sentence models based on the one or more words, wherein each sentence model corresponds to a sentence in the conversation, and wherein each sentence model provides one or more key phrases for each sentence;

generating a combined model that associates one or more partitions of the graph model and the one or more key phrases with one or more portions of the conversation stream based on a correlation between the one or more key phrases and the one or more partitions of the graph model;

employing the combined model to determine one or more dominance scores for the one or more portions of the conversation stream based on a strength of the correlation between the one or more key phrases and the one or more partitions of the graph model; and

determining one or more context switches in the conversation based on the one or more dominance scores, wherein a transition from a portion of the conversation stream associated with a dominance score that exceeds a threshold value to another portion of the conversation stream associated with another dominance score that is below another threshold values corresponds to each context switch, and wherein the one or more context switches are employed to identify one or more of topics or sub-topics in the conversation.

2. The method of claim 1 , wherein generating the graph model, further comprises:

determining one or more characteristics of the one or more words, wherein the one or more characteristics include one or more natural language processing (NLP) features, including one or more of frequency of occurrence in the conversation, or time of occurrence in the conversation;

determining the one or more relationships between the one or more words based on one or more of occurrence in a same sentence, a temporal distance in the conversation, or an occurrence in nearby sentences;

associating the one or more words with one or more nodes in the graph model based on the one or more characteristics of the one or more words; and

generating the one or more edges in the graph model based on the one or more relationships.

3. The method of claim 1 , further comprising:

determining one or more relationship scores for the one or more relationships between the one or more words in the graph model;

associating each relationship score with an edge in the graph model that corresponds to the one or more relationships between each word; and

partitioning the graph model based on one or more attributes of the nodes and one or more edges in the graph model, wherein nodes associated with the one or more relationship scores that exceed a threshold value are grouped into a same partition.

4. The method of claim 1 , wherein providing the one or more words, further comprises, providing the one or more words from one or more of a live conversation, a recorded conversation, a live video, a recorded video, or a transcription of the conversation, wherein speech in an audio portion of the live conversation, the recorded conversation, the live video, or the recorded video is converted into text.

5. The method of claim 1 , wherein generating the combined model, further comprises:

generating an adjacency matrix that include one or more rows based on the graph model to represent the conversation, wherein the one or more rows of the adjacency matrix corresponds to the one or more portions of the conversation stream;

modifying the adjacency matrix based on the sentence models to include one or more phrasal scores associated with the one or more key phrases in the conversation, wherein the one or more phrasal scores are included in the one or more rows that correspond to the one or more portions of the conversation stream at one or more positions in the conversation stream where the one or more key phrases occur in the conversation;

determining one or more eigen values associated with the one or more portions of conversation stream based on the adjacency matrix; and

employing the one or more eigen values to determine the one or more dominance scores associated with the one or more portions of the conversation stream.

6. The method of claim 1 , wherein generating the one or more sentence models, further comprises:

providing one or more sentence words from the one or more words provided by the conversation stream based on natural language processing, wherein one or more stop words may be excluded from the one or more sentence words;

determining one or more phrase scores for the one or more sentence words based on a scoring model, wherein the scoring model determines the one or more phrase scores based on one or more of repetition of a di-gram in the sentence, a word-degree, or a length of a phrase scaled the word-degree of the sentence; and

determining the one or more key phrases in one or more sentence words based on the one or more phrase scores associated with the one or more key phrases exceeding a threshold value.

7. A system for organizing conversation information over a network, comprising:

a network computer, comprising:

a memory that stores at least instructions; and

one or more processors that execute instructions that perform actions, including:

providing a graph model for a conversation based on one or more words provided by a conversation stream, wherein each word is associated with a node in the graph model and each edge in the graph model corresponds to one or more relationships between each word;

generating one or more sentence models based on the one or more words, wherein each sentence model corresponds to a sentence in the conversation, and wherein each sentence model provides one or more key phrases for each sentence;

generating a combined model that associates one or more partitions of the graph model and the one or more key phrases with one or more portions of the conversation stream based on a correlation between the one or more key phrases and the one or more partitions of the graph model;

employing the combined model to determine one or more dominance scores for the one or more portions of the conversation stream based on a strength of the correlation between the one or more key phrases and the one or more partitions of the graph model; and

determining one or more context switches in the conversation based on the one or more dominance scores, wherein a transition from a portion of the conversation stream associated with a dominance score that exceeds a threshold value to another portion of the conversation stream associated with another dominance score that is below another threshold values corresponds to each context switch; and

a client computer, comprising:

another memory that stores at least instructions; and

one or more processors that execute instructions that perform actions, including:

providing one or more of the conversation stream or the one or more words to the network computer.

8. The system of claim 7 , wherein generating the graph model, further comprises:

determining one or more characteristics of the one or more words, wherein the one or more characteristics include one or more natural language processing (NLP) features, including one or more of frequency of occurrence in the conversation, or time of occurrence in the conversation;

determining the one or more relationships between the one or more words based on one or more of occurrence in the same sentence, temporal distance in the conversation, or occurrence in nearby sentences;

associating the one or more words with one or more nodes in the graph model based on the one or more characteristics of the one or more words; and

generating the one or more edges in the graph model based on the one or more relationships.

9. The system of claim 7 , further comprising:

determining one or more relationship scores for the one or more relationships between the one or more words in the graph model;

associating each relationship score with the edge in the graph model that corresponds to the one or more relationships between each word; and

partitioning the graph model based on one or more attributes of the nodes and one or more edges in the graph model, wherein nodes associated with the one or more relationship scores that exceed a threshold value are grouped into a same partition.

10. The system of claim 7 , wherein providing the one or more words, further comprises, providing the one or more words from one or more of a live conversation, a recorded conversation, a live video, a recorded video, or a transcription of the conversation, wherein the live conversation, the recorded conversation, the live video, or the recorded video are automatically converted from speech to text.

11. The system of claim 7 , wherein generating the combined model, further comprises:

generating an adjacency matrix that include one or more rows based on the graph model to represent the conversation, wherein the one or more rows of the adjacency matrix corresponds to the one or more portions of the conversation stream;

modifying the adjacency matrix based on the sentence models to include one or more phrasal scores associated with the one or more key phrases in the conversation, wherein the one or more phrasal scores are included in the one or more rows that correspond to the one or more portions of the conversation stream where the one or more key phrases occur in the conversation;

determining one or more eigen values associated with the one or more portions of conversation stream based on the adjacency matrix; and

employing the one or more eigen values to determine the one or more dominance scores associated with the one or more portions of the conversation stream.

12. The system of claim 7 , wherein generating the one or more sentence models, further comprises:

providing one or more sentence words from the one or more words provided by the conversation stream based on natural language processing, wherein one or more stop words may be excluded from the one or more sentence words;

determining one or more phrase scores for the one or more sentence words based on a scoring model, wherein the scoring model determines the one or more phrase scores based on one or more of repetition of diagrams in the entire sentence, word-degree, or length of the entire phrase scaled with respect to the word degree in the entire sentence; and

determining the one or more key phrases in one or more sentence words based on based on the one or more phrase scores associates with the key phrases exceeding a threshold value.

13. A processor readable non-transitory storage media that includes instructions for organizing conversation information over a network, wherein execution of the instructions by one or more hardware processors performs actions, comprising:

providing a graph model for a conversation based on one or more words provided by a conversation stream, wherein each word is associated with a node in the graph model and each edge in the graph model corresponds to one or more relationships between each word;

generating one or more sentence models based on the one or more words, wherein each sentence model corresponds to a sentence in the conversation, and wherein each sentence model provides one or more key phrases for each sentence;

generating a combined model that associates one or more partitions of the graph model and the one or more key phrases with one or more portions of the conversation stream based on a correlation between the one or more key phrases and the one or more partitions of the graph model;

employing the combined model to determine one or more dominance scores for the one or more portions of the conversation stream based on a strength of the correlation between the one or more key phrases and the one or more partitions of the graph model; and

determining one or more context switches in the conversation based on the one or more dominance scores, wherein a transition from a portion of the conversation stream associated with a dominance score that exceeds a threshold value to another portion of the conversation stream associated with another dominance score that is below another threshold values corresponds to each context switch.

14. The media of claim 13 , wherein generating the graph model, further comprises:

determining one or more characteristics of the one or more words, wherein the one or more characteristics include one or more natural language processing (NLP) features, including one or more of frequency of occurrence in the conversation, or time of occurrence in the conversation;

determining the one or more relationships between the one or more words based on one or more of occurrence in the same sentence, temporal distance in the conversation, or occurrence in nearby sentences;

associating the one or more words with one or more nodes in the graph model based on the one or more characteristics of the one or more words; and

generating the one or more edges in the graph model based on the one or more relationships.

15. The media of claim 13 , further comprising:

determining one or more relationship scores for the one or more relationships between the one or more words in the graph model;

associating each relationship score with the edge in the graph model that corresponds to the one or more relationships between each word;

partitioning the graph model based on one or more attributes of the nodes and one or more edges in the graph model, wherein nodes associated with the one or more relationship scores that exceed a threshold value are grouped into a same partition.

16. The media of claim 13 , providing the one or more words, further comprises, providing the one or more words from one or more of a live conversation, a recorded conversation, a live video, a recorded video, or a transcription of the conversation, wherein the live conversation, the recorded conversation, the live video, or the recorded video are automatically converted from speech to text.

17. The media of claim 13 , wherein generating the combined model, further comprises:

generating an adjacency matrix that include one or more rows based on the graph model to represent the conversation, wherein the one or more rows of the adjacency matrix corresponds to the one or more portions of the conversation stream;

modifying the adjacency matrix based on the sentence models to include one or more phrasal scores associated with the one or more key phrases in the conversation, wherein the one or more phrasal scores are included in the one or more rows that correspond to the one or more portions of the conversation stream where the one or more key phrases occur in the conversation;

determining one or more eigen values associated with the one or more portions of conversation stream based on the adjacency matrix; and

employing the one or more eigen values to determine the one or more dominance scores associated with the one or more portions of the conversation stream.

18. The media of claim 13 , wherein generating the one or more sentence models, further comprises:

providing one or more sentence words from the one or more words provided by the conversation stream based on natural language processing, wherein one or more stop words may be excluded from the one or more sentence words;

determining one or more phrase scores for the one or more sentence words based on a scoring model, wherein the scoring model determines the one or more phrase scores based on one or more of repetition of diagrams in the entire sentence, word-degree, or length of the entire phrase scaled with respect to the word degree in the entire sentence; and

determining the one or more key phrases in one or more sentence words based on based on the one or more phrase scores associates with the key phrases exceeding a threshold value.

19. A network computer for organizing conversation information over a network, comprising:

a memory that stores at least instructions; and

one or more processors that execute instructions that perform actions, including:

providing a graph model for a conversation based on one or more words provided by a conversation stream, wherein each word is associated with a node in the graph model and each edge in the graph model corresponds to one or more relationships between each word;

generating one or more sentence models based on the one or more words, wherein each sentence model corresponds to a sentence in the conversation, and wherein each sentence model provides one or more key phrases for each sentence;

generating a combined model that associates one or more partitions of the graph model and the one or more key phrases with one or more portions of the conversation stream based on a correlation between the one or more key phrases and the one or more partitions of the graph model;

employing the combined model to determine one or more dominance scores for the one or more portions of the conversation stream based on a strength of the correlation between the one or more key phrases and the one or more partitions of the graph model; and

determining one or more context switches in the conversation based on the one or more dominance scores, wherein a transition from a portion of the conversation stream associated with a dominance score that exceeds a threshold value to another portion of the conversation stream associated with another dominance score that is below another threshold values corresponds to each context switch.

20. The network computer of claim 19 , wherein generating the graph model, further comprises:

determining one or more characteristics of the one or more words, wherein the one or more characteristics include one or more natural language processing (NLP) features, including one or more of frequency of occurrence in the conversation, or time of occurrence in the conversation;

determining the one or more relationships between the one or more words based on one or more of occurrence in the same sentence, temporal distance in the conversation, or occurrence in nearby sentences;

associating the one or more words with one or more nodes in the graph model based on the one or more characteristics of the one or more words; and

generating the one or more edges in the graph model based on the one or more relationships.

21. The network computer of claim 19 , further comprising:

determining one or more relationship scores for the one or more relationships between the one or more words in the graph model;

associating each relationship score with the edge in the graph model that corresponds to the one or more relationships between each word; and

partitioning the graph model based on one or more attributes of the nodes and one or more edges in the graph model, wherein nodes associated with the one or more relationship scores that exceed a threshold value are grouped into a same partition.

22. The network computer of claim 19 , wherein providing the one or more words, further comprises, providing the one or more words from one or more of a live conversation, a recorded conversation, a live video, a recorded video, or a transcription of the conversation, wherein the live conversation, the recorded conversation, the live video, or the recorded video are automatically converted from speech to text.

23. The network computer of claim 19 , wherein generating the combined model, further comprises:

generating an adjacency matrix that include one or more rows based on the graph model to represent the conversation, wherein the one or more rows of the adjacency matrix corresponds to the one or more portions of the conversation stream;

modifying the adjacency matrix based on the sentence models to include one or more phrasal scores associated with the one or more key phrases in the conversation, wherein the one or more phrasal scores are included in the one or more rows that correspond to the one or more portions of the conversation stream where the one or more key phrases occur in the conversation;

determining one or more eigen values associated with the one or more portions of conversation stream based on the adjacency matrix; and

employing the one or more eigen values to determine the one or more dominance scores associated with the one or more portions of the conversation stream.

24. The network computer of claim 19 , wherein generating the one or more sentence models, further comprises:

providing one or more sentence words from the one or more words provided by the conversation stream based on natural language processing, wherein one or more stop words may be excluded from the one or more sentence words;

determining one or more phrase scores for the one or more sentence words based on a scoring model, wherein the scoring model determines the one or more phrase scores based on one or more of repetition of diagrams in the entire sentence, word-degree, or length of the entire phrase scaled with respect to the word degree in the entire sentence; and

determining the one or more key phrases in one or more sentence words based on based on the one or more phrase scores associates with the key phrases exceeding a threshold value.

Assignments (5)
RELEASE OF SECURITY INTEREST Recorded Oct 31, 2025
From: FIRST-CITIZENS BANK & TRUST COMPANY
To: RAMMER TECHNOLOGIES, INC.
Reel/Frame 072752/0079 →
NUNC PRO TUNC ASSIGNMENT Recorded Jul 8, 2025
From: RAMMER TECHNOLOGIES, INC.
To: INVOCA, INC.
Reel/Frame 071629/0933 →
SECURITY INTEREST Recorded Aug 19, 2024
From: RAMMER TECHNOLOGIES, INC.
To: SILICON VALLEY BANK, A DIVISION OF FIRST-CITIZENS BANK & TRUST COMPANY
Reel/Frame 068328/0499 →
SECURITY INTEREST Recorded Jun 7, 2024
From: RAMMER TECHNOLOGIES, INC.
To: SILICON VALLEY BANK, A DIVISION OF FIRST-CITIZENS BANK & TRUST COMPANY
Reel/Frame 067650/0542 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: JAWALE, TOSHISH ARUN; BABU, ANSUP; CLAUDIA, ANTHONY
To: RAMMER TECHNOLOGIES, INC.
Reel/Frame 056104/0371 →
Continuity (1)
Provisional Application 63119957 · Dec 1, 2020