IP Library Granted Patent US 12,014,144
Granted Patent B2
US 12,014,144 · App. 17/390,573 · Granted Jun 18, 2024

Methods and systems for automatic call data generation

Inventors: Zhewen Fan (San Diego, CA); Byungkyu Kang (San Diego, CA); Wan Yu Zhang (Mountain View, CA); Carlos A. Oliveira (Mountain View, CA); Wenxin Xiao (Mountain View, CA)
Assignee: INTUIT INC.
G06F40/30G06F16/38G06F18/22G06F40/279G10L15/19G10L15/22H04M3/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,014,144
App. No.
17/390,573
Granted
Jun 18, 2024
Kind
B2
Abstract

A processor may receive a call transcript including text and form a text string including at least a portion of the text. The processor may generate a situation description of the call transcript, which may comprise processing the text string using a transformer-based machine learning model. The processor may generate a trouble description of the call transcript, which may comprise creating a sentence embedding of the situation description, creating sentence embeddings for a plurality of utterances within the portion of the text, determining respective similarities between the sentence embedding of the situation description and each of the sentence embeddings for each respective one of the plurality of utterances, and selecting at least one of the plurality of utterances having at least one highest determined respective similarity as the trouble description. The processor may store a call summary comprising the situation description and the trouble description in a non-transitory memory.

Claims (69)

1. A method comprising:

receiving, by a processor, a call transcript including text;

forming, by the processor, a text string including at least a portion of the text;

generating, by the processor, a situation description of the call transcript, the generating of the situation description comprising processing the text string using a fine-tuned transformer-based machine learning model;

generating, by the processor, a trouble description of the call transcript, the generating of the trouble description comprising:

creating, using a sentence transformer algorithm, a sentence embedding of the situation description,

creating, using the sentence transformer algorithm, sentence embeddings for a plurality of utterances within the portion of the text,

determining respective similarities between the sentence embedding of the situation description and each of the sentence embeddings for each respective one of the plurality of utterances, and

selecting at least one of the plurality of utterances having at least one highest determined respective similarity as the trouble description; and

storing, by the processor, a call summary comprising the situation description and the trouble description in a non-transitory memory accessible to at least one call-handling system, wherein the fine-tuned transformer-based machine learning model is fine tuned by a process comprising:

receiving the transformer-based machine learning model in a pretrained state wherein the transformer-based machine learning model has been previously trained with generic text,

generating labeled call summaries by receiving unlabeled call summaries, selecting at least one of the unlabeled call summaries starting with a keyword or keyphrase, and selecting respective excerpts of each of the selected at least one of the unlabeled call summaries for inclusion within the labeled call summaries, and

performing further training on the transformer-based machine learning model in the pretrained state using the labeled call summaries.

2. The method of claim 1 , further comprising:

generating, by the processor, the call transcript, the generating of the call transcript comprising:

selecting a subset of an audio recording of a call as a reduced portion of the audio recording, and

automatically transcribing only the reduced portion of the audio recording.

3. The method of claim 2 , wherein the identifying comprises selecting a most recent portion of a predetermined length of the audio recording as the reduced portion or selecting an oldest portion of the predetermined length of the audio recording as the reduced portion.

4. The method of claim 1 , wherein the forming of the text string comprises adding metadata to the text string, the metadata being related to a call from which the call transcript was taken.

5. The method of claim 1 , wherein the determining of the respective similarities is performed using a pairwise cosine similarity function.

6. The method of claim 1 , further comprising:

receiving a second call;

determining, by the processor, that the second call is related to the call transcript; and

providing, by the processor, the call summary in a user interface.

7. A system comprising:

a processor; and

a non-transitory memory in communication with the processor, the non-transitory memory storing instructions that, when executed by the processor, cause the processor to perform processing comprising:

receiving a call transcript including text;

forming a text string including at least a portion of the text;

generating a situation description of the call transcript, the generating of the situation description comprising processing the text string using a fine-tuned transformer-based machine learning model;

generating a trouble description of the call transcript, the generating of the trouble description comprising:

creating, using a sentence transformer algorithm, a sentence embedding of the situation description,

creating, using the sentence transformer algorithm, sentence embeddings for a plurality of utterances within the portion of the text,

determining respective similarities between the sentence embedding of the situation description and each of the sentence embeddings for each respective one of the plurality of utterances, and

selecting at least one of the plurality of utterances having at least one highest determined respective similarity as the trouble description; and

storing a call summary comprising the situation description and the trouble description in the non-transitory memory, wherein the fine-tuned transformer-based machine learning model is fine tuned by a process comprising:

receiving the transformer-based machine learning model in a pretrained state wherein the transformer-based machine learning model has been previously trained with generic text,

generating labeled call summaries by receiving unlabeled call summaries, selecting at least one of the unlabeled call summaries starting with a keyword or keyphrase, and selecting respective excerpts of each of the selected at least one of the unlabeled call summaries for inclusion within the labeled call summaries, and

performing further training on the transformer-based machine learning model in the pretrained state using the labeled call summaries.

8. The system of claim 7 , wherein the processing further comprises:

generating the call transcript, the generating of the call transcript comprising:

selecting a subset of an audio recording of a call as a reduced portion of the audio recording, and

automatically transcribing only the reduced portion of the audio recording.

9. The system of claim 8 , wherein the identifying comprises selecting a most recent portion of a predetermined length of the audio recording as the reduced portion or selecting an oldest portion of the predetermined length of the audio recording as the reduced portion.

10. The system of claim 7 , wherein the forming of the text string comprises adding metadata to the text string, the metadata being related to a call from which the call transcript was taken.

11. The system of claim 7 , wherein the determining of the respective similarities is performed using a pairwise cosine similarity function.

12. The system of claim 7 , wherein the processing further comprises:

receiving a second call;

determining that the second call is related to the call transcript; and

providing the call summary in a user interface.

13. A method comprising:

receiving, at a processor, a transformer-based machine learning model in a pretrained state wherein the transformer-based machine learning model has been previously trained with generic text;

generating, by the processor, labeled call summaries, the generating of the labeled call summaries comprising:

receiving unlabeled call summaries,

selecting at least one of the unlabeled call summaries starting with a keyword or keyphrase, and

selecting respective excerpts of each of the selected at least one of the unlabeled call summaries for inclusion within the labeled call summaries;

performing, by the processor, further training on the transformer-based machine learning model in the pretrained state using the labeled call summaries;

receiving, at the processor, a call transcript including text;

forming, by the processor, a text string including at least a portion of the text;

generating, by the processor, a situation description of the call transcript, the generating of the situation description comprising processing the text string using the transformer-based machine learning model; and

storing, by the processor, a call summary comprising the situation description in a non-transitory memory accessible to at least one call-handling system.

14. The method of claim 13 , further comprising:

generating, by the processor, a trouble description of the call transcript, the generating of the trouble description comprising:

creating, using a sentence transformer algorithm, a sentence embedding of the situation description,

creating, using the sentence transformer algorithm, sentence embeddings for a plurality of utterances within the portion of the text,

determining respective similarities between the sentence embedding of the situation description and each of the sentence embeddings for each respective one of the plurality of utterances, and

selecting at least one of the plurality of utterances having at least one highest determined respective similarity as the trouble description; and

including, by the processor, the trouble description in the call summary stored in the non-transitory memory.

15. The method of claim 14 , wherein the determining of the respective similarities is performed using a pairwise cosine similarity function.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 2, 2021
From: FAN, ZHEWEN; KANG, BYUNGKYU; ZHANG, WAN YU; OLIVEIRA, CARLOS A.; XIAO, WENXIN
To: INTUIT INC.
Reel/Frame 057376/0800 →
Continuity (1)
Related Publication 20230030405A1 · Feb 2, 2023
Cited By (1)
US 12,400,072