Method and apparatus for speech behavior visualization and gamification
View Patent ↗In some example embodiments, a system is provided for real-time analysis of audio signals. First digital audio signals are retrieved from memory. First computed streamed signal information corresponding to each of the first digital audio signals is generated by computing first metrics data for the first digital audio signals, the first computed streamed signal information including the first metrics data. The computed first streamed signal information is stored in the memory. The first computed streamed signal information is transmitted to one or more computing devices. Transmitting the first computed streamed signal information to the one or more computing devices causes the first computed streamed signal information to be displayed at the one or more computing devices.
1. A method for real-time graphical presentation of speech audio signal analytics of a multi-party telephonic communication for feedback to at least one party, the method comprising:
computing, by a processor of a computing device, on a continuous basis during the multi-party telephonic communication between at least a first party and a second party, a set of metrics from a digital audio signal corresponding to the multi-party telephonic communication, the set of metrics comprising at least one of Group A and at least two of Group B, wherein Group A consists of:
(i) a measure of tone with which the first party has spoken and a measure of tone with which the second party has spoken, over an interval of time;
(ii) a measure of speaking rate at which the first party has spoken and a measure of speaking rate at which the second party has spoken, over an interval of time;
(iii) a measure of vocal effort with which the first party has spoken and a measure of vocal effort with which the second party has spoken, over an interval of time; and
(iv) a measure of degree of articulation with which the first party has spoken and a measure of degree of articulation with which the second party has spoken, over an interval of time; and
Group B consists of:
(v) a measure of amount of time the first party has spoken relative to the second party, over an interval of time;
(vi) a measure of conversational engagement of the parties, over an interval of time;
(vii) a measure of perceived depression with which a party has spoken, over an interval of time; and
(viii) a measure of conversational flow over an interval of time, wherein the measure of conversational flow indicates smoothness and connection of turn-taking between participants in a conversation; and
rendering, by a processor of a computing device, on a continuous basis during the multi-party telephonic communication, one or more graphical user interface widgets for substantially contemporaneous presentation on a display to the first party, wherein the one or more widgets are graphically representative of the set of metrics and are rendered for display on a real-time basis, wherein the real-time rendering for display is characterized by rendering at low-latency.
2. The method of claim 1 , wherein the rendering step comprises rendering a timeline widget that scrolls contemporaneously with at least a portion of the telephonic communication graphically indicating when the first party is speaking and when the second party is speaking.
3. The method of claim 1 , wherein the rendering step comprises rendering a numerical representation of the measure of conversational engagement at intervals contemporaneously with at least a portion of the telephonic communication, leaving behind a graphical record of engagement scores corresponding to intervals of time during the telephonic communication.
4. The method of claim 3 , wherein each rendering of the engagement scores is color-coded such that low engagement scores can be immediately visually differentiated from high engagement scores.
5. The method of claim 1 , wherein the rendering step comprises:
rendering a tone widget comprising a graphical element representing the measure of tone of the first party in positional relation to a graphical element representing the measure of tone of the second party, and
updating the widget substantially contemporaneously with the telephonic communication to reflect changes in the measures of tone of the first party and the second party.
6. The method of claim 5 , wherein the tone widget graphically reflects both an instantaneous measure of tone of the first party and the second party, and a rolling measure of tone of the first party and the second party.
7. The method of claim 1 , wherein the rendering step comprises
rendering a pace widget comprising a graphical element representing the measure of speaking rate of the first party in positional relation to a graphical element representing the measure of speaking rate of the second party, and
updating the widget substantially contemporaneously with the telephonic communication to reflect changes in the measures of speaking rate of the first party and the second party.
8. The method of claim 7 , wherein the pace widget graphically reflects both an instantaneous measure of speaking rate of the first party and the second party, and a rolling measure of speaking rate of the first party and the second party.
9. The method of claim 1 , wherein the rendering step comprises rendering a participation widget comprising a graphical element representing the measure of amount of time the first party has spoken relative to the second party over an interval of time.
10. The method of claim 9 , wherein the participation widget comprises a color-coded graphic visually indicating whether the measure of amount of time the first party has spoken relative to the second party over the interval of time is acceptable or not.
11. The method of claim 1 , wherein the set of metrics comprises at least two of Group A and two of Group B.