Transcript tagging and real-time whisper in interactive communications
Disclosed herein are system, method, and computer readable medium embodiments for machine learning systems to process interactive communications between at least two participants. Speech and text within the interactive communications are analyzed using machine learning models to infer insights located within the interactive communications. The inferred insights are converted to descriptive text or audio and tagged to the interactive communication as graphics or audio whispers reflecting the insights added to the interactive communication.
1. A system for augmenting an interactive communication in a natural language processing environment, the system configured to:
convert, by a first machine learning model trained by a machine learning system, the interactive communication to a textual transcript;
extract and evaluate, by a second machine learning model trained by the machine learning system, textual cues in the interactive communication to infer a first insight located within the textual transcript;
generate first text corresponding to a description of the first insight;
tag a first section within the interactive communication with the first text corresponding to the description of the first insight;
display the first text proximate to the tagged first section of a graphic waveform of the interactive communication;
extract and evaluate, by a third machine learning model, audio cues in the interactive communication to infer a second insight located within the interactive communication;
generate second text corresponding to a description of the second insight;
tag a second section within the interactive communication with the second text corresponding to the description of the second insight;
display the second text proximate to the tagged second section of the graphic waveform of the interactive communication;
generate a first audio instance of the first text representing the first insight;
generate a second audio instance of the second text representing the second insight; and
overlay the first audio instance on the tagged first section as a first whisper voice and the second audio instance on the tagged second section as a second whisper voice of the interactive communication.
2. The system of claim 1 further configured to:
tag the first section within the textual transcript with the second text corresponding to the description of the second insight; and
display a graphic with the second text proximate to the first section of the textual transcript.
3. The system of claim 1 , wherein the third machine learning model comprises:
a prosodic cue model to extract and evaluate prosodic cues within the interactive communication.
4. The system of claim 3 , wherein the prosodic cues comprise any of:
frequency changes, pitch, pauses, length of sounds, volume, loudness, speech rate, voice quality, or stress placed on a specific utterance of speech.
5. The system of claim 1 further configured to:
display a graphic of the first text or the second text proximate to a selected section of the textual transcript.
6. The system of claim 1 further configured to:
superimpose the first audio instance or the second audio instance proximate to a selected section of the interactive communication.
7. The system of claim 1 , wherein the second machine learning model comprises any of:
a sentiment predictive model to extract and evaluate semantic cues within the textual transcript;
a key word model to extract and evaluate key words within the textual transcript;
a complaint predictive model to extract and evaluate semantic and key words within the textual transcript; or
a disclosure compliance predictive model to extract and evaluate disclosure key words within the textual transcript.
8. A computer-implemented method for processing a call in a natural language environment, comprising:
converting, by a first machine learning model trained by a machine learning system, an interactive communication to a textual transcript;
extracting and evaluating, by a second machine learning model trained by the machine learning system, textual cues in the interactive communication to infer a first insight located within the textual transcript;
generating first text corresponding to a description of the first insight;
tagging a first section within the interactive communication with the first text;
displaying the first text proximate to the tagged first section of a graphic waveform of the interactive communication;
extracting and evaluating, by a third machine learning model trained by the machine learning system, audio cues in the interactive communication to infer a second insight located within the interactive communication;
generating second text corresponding to a description of the second insight;
tagging a second section within the interactive communication with the second text corresponding to the description of the second insight;
displaying the second text proximate to the tagged second section of the graphic waveform of the interactive communication;
generating a first audio instance of the first text representing the first insight;
generating a second audio instance of the second text representing the second insight; and
overlaying the first audio instance proximate to the tagged first section as a first whisper voice and the second audio instance proximate to a tagged second section as a second whisper voice of the interactive communication.
9. The computer-implemented method of claim 8 , wherein the third machine learning model comprises a prosodic cue model to extract and evaluate prosodic cues within the interactive communication.
10. The computer-implemented method of claim 8 further comprising
displaying a graphic with the second text proximate to the second section of the textual transcript.
11. The computer-implemented method of claim 8 wherein the second machine learning model comprises any of:
a sentiment predictive model to extract and evaluate semantic cues within the textual transcript;
a key word model to extract and evaluate key words within the textual transcript;
a complaint predictive model to extract and evaluate semantic and key words within the textual transcript; or
a disclosure compliance predictive model to extract and evaluate disclosure key words within the textual transcript.
12. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform natural language operations comprising:
converting, by a first machine learning model trained by a machine learning system, an interactive communication to a textual transcript;
extracting and evaluating, by a second machine learning model trained by the machine learning system, textual cues in the interactive communication to infer a first insight located within the textual transcript;
generating first text corresponding to a description of the first insight;
tagging a first section within the interactive communication with the first text;
displaying the first text proximate to the tagged first section of a graphic waveform of the interactive communication;
extracting and evaluating, by a third machine learning model trained by the machine learning system, audio cues in the interactive communication to infer a second insight located within the interactive communication;
generating second text corresponding to a description of the second insight;
tagging a second section within the interactive communication with the second text; and
displaying the second text proximate to the tagged second section of the graphic of the waveform of the interactive communication;
generating a first audio instance of the first text representing the first insight;
generating a second audio instance of the second text representing the second insight; and
overlaying the first audio instance proximate to the tagged first section as a first whisper voice and the second audio instance proximate to the tagged second section as a second whisper voice of the interactive communication.