Granted Patent B2

US 12,314,667 · App. 17/719,314 · Granted May 27, 2025

Inventors: Peng Su (Shoreline, WA); Ling Tsou (Los Angeles, CA); Junqing Wang (Zhejiang, CN); Tong Wang (Lexington, MA)

Assignee: Zoom Communications, Inc.

G06F40/295G06F40/205G06F40/40G06N20/00H04L12/1831

Abstract

Claims (70)

1. A method, comprising:

connecting, by a server-side processing system, to a communication session including one or more client devices hosted on a network;

receiving, by the server-side processing system, a plurality of utterances captured by the one or more client devices and associated with speaking participants of the communication session;

generating, by the server-side processing system, a transcript including the plurality of utterances captured by the one or more client devices, the transcript comprising, for the plurality of utterances, indications of a speaking participant and timestamps;

obtaining, by the server-side processing system, a machine learning (ML) model trained using topic modeling, wherein topic modeling defines topic model (TM) clusters in the communication session based on a training corpus comprising transcripts of past communication sessions;

using, by the server-side processing system, the ML model to generate a plurality of blocks of utterances by shifting a window over the plurality of utterances, the window having a window size of a prespecified number of utterances and a step size of one utterance;

for each shift of the window over one utterance:

identifying a plurality of TM clusters; and

generating, via the ML model, TM cluster scores to assign text labels for each TM cluster;

aggregating, by the server-side processing system, the TM cluster scores to generate a plurality of utterance level scores;

associating, by the server-side processing system, each utterance level score with a corresponding timestamp;

generating, by the server-side processing system based on the utterance level scores that exceed a threshold value, topic segments each comprising a start time, end time, and associated text label, the start time and the end time being determined based on corresponding timestamps of the utterance level scores; and

transmitting, by the server-side processing system to a client device of the one or more client devices based on a permission level associated with the client device, a list of the topic segments for the communication session.

2. The method of claim 1 , wherein the text labels for each TM cluster are assigned based on a top phrasal feature of the TM cluster, the top phrasal feature being determined based on the TM cluster score for the TM cluster.

3. The method of claim 1 , wherein the ML model is trained with Term Frequency-Inverse Document Frequency (TF-IDF) to compute a vector representation for each block of utterances.

4. The method of claim 1 , wherein the text labels are predefined.

5. The method of claim 1 , wherein one or more of Non-Negative Matrix Factorization (NMF) and Latent Dirichlet Allocation (LDA) are used as topic modeling methods.

6. The method of claim 1 , further comprising parsing and tagging words in the transcript as parts of speech (POS) via a pretrained natural language processing (NLP) model; and

training the ML model using the parsed and tagged words in the transcript.

7. The method of claim 1 , wherein Named Entity Recognition (NER) is used to determine one or more named entities and assign symbolic representations to the named entities.

8. The method of claim 1 , wherein generating the TM cluster scores comprises using topic modeling inferencing techniques.

9. The method of claim 1 , wherein aggregating the TM cluster scores to generate the plurality of utterance level scores comprises:

for each utterance:

identifying blocks of utterances that include a current utterance;

identifying a TM cluster with a highest score from each block of utterances to generate a set of TM clusters; and

generating an utterance level score for the utterance based on the TM cluster scores of the generated set of TM clusters.

10. The method of claim 1 , wherein generating the topic segments comprises grouping utterances of a same level together.

11. The method of claim 1 , further comprising performing one or more smoothing and/or adjustment operations after the utterance level scores are generated.

12. The method of claim 11 , wherein at least one adjustment operation comprises assigning an ambiguous TM cluster to a predefined label based on a position of a topic segment within the communication session.

13. The method of claim 11 , wherein at least one adjustment operation comprises assigning a TM cluster to a predefined label based on which participant is speaking with a highest frequency.

14. The method of claim 11 , wherein at least one smoothing operation comprises:

determining that a topic segment does not meet a threshold duration requirement; and

merging the topic segment with one or both of its neighboring topic segments.

15. The method of claim 11 , wherein at least one adjustment operation comprises:

discounting and removing an utterance level score if it does not meet a threshold value.

16. The method of claim 1 , further comprising:

prior to transmitting the list of the topic segments to the client devices, formatting the list of topic segments into a structured data markup format.

17. A server-side communication system comprising one or more processors configured to perform operations of:

connecting to a communication session including one or more client devices hosted on a network;

receiving a plurality of utterances captured by the one or more client devices and associated with speaking participants of the communication session;

generating a transcript including the plurality of utterances captured by the one or more client devices, the transcript comprising, for the plurality of utterances, indications of a speaking participant and timestamps;

obtaining a machine learning (ML) model trained using topic modeling, wherein topic modeling defines topic model (TM) clusters in the communication session based on a training corpus comprising transcripts of past communication sessions;

training a machine learning (ML) model using topic modeling, wherein topic modeling defines topic model (TM) clusters in the communication session based on a training corpus comprising transcripts of past communication sessions;

using the ML model to generate a plurality of blocks of utterances by shifting a window over the plurality of utterances, the window having a window size of a prespecified number of utterances and a step size of one utterance;

for each shift of the window over one utterance:

identifying a plurality of TM clusters; and

generating, via the ML model, TM cluster scores to assign text labels for each TM cluster;

aggregating the TM cluster scores to generate a plurality of utterance level scores;

associating each utterance level score with a corresponding timestamp;

generating, based on the utterance level scores that exceed a threshold value, topic segments each comprising a start time, end time, and associated text label, the start time and the end time being determined based on corresponding timestamps of the utterance level scores; and

transmitting, to a client device of the one or more client devices based on a permission level associated with the client device, a list of the topic segments for the communication session.

18. The server-side communication system of claim 17 , wherein:

the communication session is a sales session with one or more prospective customers and one or more members of a sales team, and

the text labels for each TM cluster and topic segments relate to the sales session, and further comprising:

identifying each of the participants of the communication session as a prospective customer or a member of the sales team.

19. The server-side communication system of claim 17 , wherein the one or more processors are further configured to perform an operation of:

allowing, via one or more authorized client devices, modification of one or more of the start time, end time, and associated TM label for at least a subset of the topic segments.

20. A non-transitory computer-readable medium storing instructions which, when executed by one or more computers, cause the one or more computers to perform operations comprising:

connecting, by a server-side processing system, to a communication session including one or more client devices hosted on a network;

receiving, by the server-side processing system, a plurality of utterances captured by the one or more client devices and associated with speaking participants of the communication session;

generating, by the server-side processing system, a transcript including the plurality of utterances captured by the one or more client devices, the transcript comprising, for the plurality of utterances, indications of a speaking participant and timestamps;

obtaining, by the server-side processing system, a machine learning (ML) model trained using topic modeling, wherein topic modeling defines topic model (TM) clusters in the communication session based on a training corpus comprising transcripts of past communication sessions;

using, by the server-side processing system, the ML model to generate a plurality of blocks of utterances by shifting a window over the plurality of utterances, the window having a window size of a prespecified number of utterances and a step size of one utterance;

for each shift of the window over one utterance:

identifying a plurality of TM clusters; and

generating, via the ML model, TM cluster scores to assign text labels for each TM cluster;

aggregating, by the server-side processing system, the TM cluster scores to generate a plurality of utterance level scores;

associating, by the server-side processing system, each utterance level score with a corresponding timestamp;

generating, by the server-side processing system based on the utterance level scores that exceed a threshold value, topic segments each comprising a start time, end time, and associated text label, the start time and the end time being determined based on corresponding timestamps of the utterance level scores; and

transmitting, by the server-side processing system to a client device of the one or more client devices based on a permission level associated with the client device, a list of the topic segments for the communication session.

Assignments (2)

Priority Claims (1)

Continuity (2)

References Cited (18)