IP Library › Granted Patent US 12,260,185
Granted Patent B2
US 12,260,185 · App. 17/159,625 · Granted Mar 25, 2025

Coarse-to-fine abstractive dialogue summarization with controllable granularity

Inventors: Chien-Sheng Wu (Mountain View, CA); Wenhao Liu (Redwood City, CA); Caiming Xiong (Menlo Park, CA); Linqing Liu (Menlo Park, CA)
Assignee: Salesforce, Inc.
G06F40/56G06F40/205
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,260,185
App. No.
17/159,625
Filed
Jan 27, 2021
Granted
Mar 25, 2025
Kind
B2
Art Unit
2144
USPC
704/9
Abstract

Dialogue summarization is challenging due to its multi-speaker standpoints, casual spoken language style, and limited labelled data. The embodiments are directed to a coarse-to-fine dialogue summarization model that improves abstractive dialogue summarization quality and enables granular controllability. A summary draft that includes key words for turns in a dialogue conversation history is created. The summary draft includes pseudo-labelled interrogative pronoun categories and noisy key phrases. The dialogue conversation history is divided into segments. A generate language model is trained to generate a segment summary for each dialogue segment using a portion of the summary draft that corresponds to at least one dialogue turn in the dialogue segment. A dialogue summary is generated using the generative language model trained using the summary draft.

Claims (68)

1. A method for training a neural network model to generate a dialogue summary, comprising:

dividing, using a similarity module executing on a processor, a dialogue conversation history into dialogue segments, wherein dividing the dialogue conversation history into the dialogue segments further comprises:

matching, using the similarity module, a plurality of dialogue segments in a dialogue conversation history against segment summaries associated with a training summary, wherein that matching generates similarity scores;

selecting the dialogue segments from the plurality of dialogue segments that correspond to highest similarity scores from the similarity scores, wherein a dialogue segment in the selected dialogue segments includes including at least one dialogue turn from dialogue turns in the dialogue conversation history;

dividing, using the similarity module, the training summary into training segment summaries based on dialogue turns in the dialogue conversation history, wherein the training summary summarizes the dialogue conversation history;

generating a summary draft from the dialogue segments in a dialogue conversation history and the training segment summaries, wherein the summary draft includes turn indexes corresponding to the dialogue turns, labels for action categories and key phrases associated with a subset of the dialogue turns;

generating, using an encoder of a generative language model executing on the processor, encodings from the dialogue segments; and

generating, using a decoder of the generative language model, segment summaries for the dialogue conversation history from the encodings from the dialogue segments and the summary draft.

2. The method of claim 1 , wherein the summary draft includes a first turn index, a label for an action category in the labels for the action categories and a key phrase in the key phrases that correspond to a first dialogue turn.

3. The method of claim 1 , wherein a label for an action category in the labels for the action categories indicates that there are no interrogative pronouns associated with a dialogue turn in the dialogue turns.

4. The method of claim 1 , wherein the summary draft is a concatenation of a first turn index corresponding to a first dialogue turn in the dialogue conversation history, a first label for a first action category in the labels for the action categories that corresponds to the first turn index, a first key phrase in the key phrases that corresponds to the first turn index, a second turn index corresponding to a second dialogue turn in the dialogue turns, a second label for a second action category in the labels for the action categories that corresponds to the second turn index, and a second key phrase in the second key phrases that corresponds to the second turn index.

5. The method of claim 1 , wherein the generating the summary draft further comprises:

generating, using a label module, at least one label for the action category in the labels for the action categories from the dialogue conversation history.

6. The method of claim 1 , wherein the generating the summary draft further comprises:

generating, using a parser, at least one key phrase in the key phrases associated with a subset of the dialogue turns and a training summary draft.

7. The method of claim 1 , further comprising:

training a dialogue turn level classifier, wherein the training comprises:

separating the dialogue turns in the dialogue conversation history with separator tokens;

classifying, using the dialogue turn level classifier, the separator tokens, wherein an output of the classifier are token vectors representing the separator tokens;

determining probabilities of the token vectors;

selecting at least one token vector from the token vectors based on at least one probability in the probabilities;

identifying at least one separator token corresponding to the at least one token vector as at least one cutting point that separates the dialogue segments in the dialogue conversation history; and

inserting at least one highlighting token in place of the at least one separator token.

8. The method of claim 7 , wherein the at least one probability is above a predefined probability threshold or further comprises selecting the at least one probability that is one of top highest probabilities.

9. The method of claim 1 , further comprising:

receiving a selection of a subset of dialogue turns from the dialogue conversation history corresponding to the dialogue segment in the dialogue segments.

10. The method of claim 1 , further comprising:

identifying, using a dialogue turn level classifier and the dialogue segments, cutting points in the dialogue conversation history, wherein one cutting point from the cutting points indicates a beginning of a dialogue segment in the dialogue segments and a second cutting point indicates an end of the dialogue segment.

11. A system for training a neural network model to generate a dialogue summary, comprising:

at least one memory configured to store a similarity module and a generative language model of the neural network model; and

at least one processor coupled to the at least one memory and configured to:

divide, using a similarity module executing on a processor, a dialogue conversation history into dialogue segments, wherein to divide the dialogue conversation history into the dialogue segments, the at least one processor is further configured to:

match, using the similarity module, a plurality of dialogue segments in a dialogue conversation history against segment summaries associated with a training summary, wherein that matching generates similarity scores;

select the dialogue segments from the plurality of dialogue segments that correspond to highest similarity scores from the similarity scores, wherein a dialogue segment in the selected dialogue segments includes at least one dialogue turn from dialogue turns in the dialogue conversation history;

divide, using the similarity module, the training summary into training segment summaries based on dialogue turns in the dialogue conversation history, wherein the training summary summarizes the dialogue conversation history;

generate a summary draft from the dialogue segments in a dialogue conversation history and the training segment summaries, wherein the summary draft includes turn indexes corresponding to the dialogue turns, labels for action categories and key phrases associated with a subset of the dialogue turns;

generate, using an encoder of a generative language model, encodings from the dialogue segments; and

generate, using a decoder of the generative language model, segment summaries for the dialogue conversation history from the encodings from the dialogue segments and the summary draft.

12. The system of claim 11 , wherein the summary draft includes a first turn index, a label for an action category in the labels for the action categories and a key phrase in the key phrases that correspond to a first dialogue turn.

13. The system of claim 11 , wherein the summary draft is a concatenation of a first turn index corresponding to a first dialogue turn in the dialogue conversation history, a first label for a first action category in the labels for the action categories that corresponds to the first turn index, a first key phrase in the key phrases that corresponds to the first turn index, a second turn index corresponding to a second dialogue turn in the dialogue turns, a second label for a second action category in the labels for the action categories that corresponds to the second turn index, and a second key phrase in the second key phrases that corresponds to the second turn index.

14. The system of claim 11 , wherein to generate the summary draft, the processor is further configured to:

generate, using a label module stored in the at least one memory, at least one label for the action category in the labels for the action categories from the dialogue conversation history.

15. The system of claim 11 , wherein the processor is further configured to:

generate, using a parser stored in the at least one memory, at least one key phrase in the key phrases associated with the subset of the dialogue turns and a training summary draft.

16. The system of claim 11 , wherein to divide the dialogue conversation history into the dialogue segments, the processor is further configured to:

separate the dialogue turns in the dialogue conversation history with separator tokens;

classify, using a dialogue turn level classifier, the separator tokens, wherein an output of the dialogue turn level classifier are token vectors representing the separator tokens;

determine probabilities of the token vectors;

select at least one token vector from the token vectors based on at least one probability in the probabilities;

identify at least one separator token corresponding to the at least one token vector as at least one cutting point that separates the dialogue segments in the dialogue conversation history; and

insert at least one highlighting token in place of the at least one separator token.

17. The system of claim 11 , further comprising:

identify, using a dialogue turn level classifier and the dialogue segments, cutting points in the dialogue conversation history, wherein one cutting point from the cutting points indicates a beginning of a dialogue segment in the dialogue segments and a second cutting point indicates an end of the dialogue segment.

18. A non-transitory computer readable medium storing instructions thereon, that when executed by a processor cause the processor to perform operations for training a neural network model to generate a dialogue summary, the operations comprising:

dividing, using a similarity module executing on a processor, a dialogue conversation history into dialogue segments, a dialogue segment including at least one dialogue turn from dialogue turns in the dialogue conversation history, wherein dividing the dialogue conversation history into the dialogue segments further comprises:

separating the dialogue turns in the dialogue conversation history with separator tokens;

classifying, using a dialogue turn level classifier, the separator tokens, wherein an output of the dialogue turn level classifier are token vectors representing the separator tokens;

determining probabilities of the token vectors;

selecting at least one token vector from the token vectors based on at least one probability in the probabilities;

identifying at least one separator token corresponding to the at least one token vector as at least one cutting point that separates the dialogue segments in the dialogue conversation history; and

inserting at least one highlighting token in place of the at least one separator token;

dividing, using the similarity module, a training summary into training segment summaries based on dialogue turns in the dialogue conversation history, wherein the training summary summarizes the dialogue conversation history;

generating a summary draft from the dialogue segments in a dialogue conversation history and the training segment summaries, wherein the summary draft includes turn indexes corresponding to the dialogue turns, labels for action categories and key phrases associated with a subset of the dialogue turns;

generating, using an encoder of a generative language model, encodings from the dialogue segments; and

generating, using a decoder of the generative language model, segment summaries for the dialogue conversation history from the encodings from the dialogue segments and the summary draft.

19. The non-transitory computer readable medium of claim 18 , wherein a label for an action category in the labels for the action categories is one of interrogative pronouns associated with a dialogue turn in the dialogue turns.

20. The non-transitory computer readable medium of claim 18 , the operations further comprise:

generating, using a parser, at least one key phrase in the key phrases associated with the subset of the dialogue turns and a training summary draft.

Assignments (2)
CHANGE OF NAME Recorded Feb 17, 2025
From: SALESFORCE.COM, INC.
To: SALESFORCE, INC.
Reel/Frame 070240/0168 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 17, 2021
From: WU, CHIEN-SHENG; LIU, WENHAO; XIONG, CAIMING; LIU, LINQING
To: SALESFORCE.COM, INC.
Reel/Frame 055294/0482 →
Continuity (2)
Provisional Application 63087024 · Oct 2, 2020
Related Publication 20220108086A1 · Apr 7, 2022
References Cited (34)
US 10977258B1 · Liu · 2021 [cited by examiner]
US 11314790B2 · Chang · 2022 [cited by examiner]
US 20140036022A1 · Croen · 2014 [cited by examiner]
US 20190325066A1 · Krishna · 2019 [cited by examiner]
US 20190327103A1 · Niekrasz · 2019 [cited by examiner]
US 20220030110A1 · Khafizov · 2022 [cited by examiner]
CN 111651996 · 2019 [cited by examiner]
CN 111651996A1 · 2019 [cited by examiner]
CN 111651996A · 2020 [cited by examiner]
WO WO2021108056A1 · 2021 [cited by examiner]
Devlin et al., “BERT: Pre-training of deep bidirectional transformers for language understanding”, arXiv preprint arXiv:1810.04805, Oct. 11, 2018, 14 pages. [cited by applicant]
Dong et al., “Unified Language Model Pre-training for Natural Language Understanding and Generation”, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada, 2019, 13 pages. [cited by applicant]
Fan et al., “Controllable Abstractive Summarization”, In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, Melbourne, Australia, Jul. 20, 2018, pp. 45-54. [cited by applicant]
Gliwa et al., “SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization”, In Proceedings of the 2nd Workshop on New Frontiers in Summarization, Hong Kong, China, Nov. 4, 2019, pp. 70-79. [cited by applicant]
Goo et al., “Abstractive Dialogue Summarization With Sentence-Gated Modeling Optimized by Dialogue Acts”, In 2018 IEEE Spoken Language Technology Workshop (SLT), Sep. 29, 2018, 8 pages. [cited by applicant]
Hermann et al., “Teaching Machines to Read and Comprehend”, In Advances in neural information processing systems, 2015, 9 pages. [cited by applicant]
Kitaev et al., “Constituency Parsing with a Self-Attentive Encoder”, In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), Melbourne, Australia, Jul. 15-20, 2018, pp. … [cited by applicant]
Lewis et al., “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension”, arXiv preprint arXiv:1910.13461, Oct. 29, 2019, 10 pages. [cited by applicant]
Liu et al., “Controlling Length in Abstractive Summarization Using a Convolutional Neural Network”, In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, Oct. 31-N… [cited by applicant]
McCowan et al., “The AMI Meeting Corpus”, Proceedings of Measuring Behavior 2005, 5th International Conference on Methods and Techniques in Behavioral Research, 2005, 4 pages. [cited by applicant]
Nallapati et al., “Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond”, Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), Berlin, Germany, Aug. 7-12, 20… [cited by applicant]
Narayan et al., “Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization”, In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processi… [cited by applicant]
Nikolov et al., “Data-driven Summarization of Scientific Articles”, arXiv preprint arXiv:1804.08875, Apr. 24, 2018, 8 pages. [cited by applicant]
Qazvinian et al., “Scientific Paper Summarization Using Citation Summary Networks”, In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), Manchester, Aug. 2008, pp. 689-696. [cited by applicant]
Radford et al., “Language Models are Unsupervised Multitask Learners”, 2019, 24 pages. [cited by applicant]
Ratner et al., “Training Complex Models with Multi-Task Weak Supervision”, The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), 2019, pp. 4763-4771. [cited by applicant]
Saito et al. “Length-controllable Abstractive Summarization by Guiding with Summary Prototype”, arXiv preprint arXiv:2001.07331, Jan. 21, 2020, 8 pages. [cited by applicant]
See et al., “Get To The Point: Summarization with Pointer-Generator Networks” arXiv preprint arXiv:1704.04368, Apr. 25, 2017, 20 pages. [cited by applicant]
Wolf et al., “TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents”, In 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada, 2018, 6 pages. [cited by applicant]
Wu et al., “TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue”, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Nov. 16-20, 2020, pp. 917-929. [cited by applicant]
Yang et al., “XLNet: Generalized Autoregressive Pretraining for Language Understanding”, In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada, 2019, 11 pages. [cited by applicant]
Zhang et al., “PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization”, In Proceedings of the 37 th International Conference on Machine Learning, Online, PMLR 119, 2020, 12 pages. [cited by applicant]
Zhang et al., “Neural Latent Extractive Document Summarization”, In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, Oct. 31-Nov. 4, 2018. pp. 779-784. [cited by applicant]
Zhang et al., “DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation”, In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul. 5-Jul. 10, 2020, pp.… [cited by applicant]