Analysis of conversational attributes with real time feedback
Devices and methods for providing real-time feedback of conversational attributes are provided. An audio signal comprising speech is received. The audio signal is divided into a plurality of sequential windows. A plurality of features is extracted from each sequential window of the audio signal. Each plurality of features is sequentially provided to a trained classifier and a speech attribute of the corresponding window of the audio signal is received therefrom. After receiving each speech attribute, and based upon that speech attribute and speech attributes of prior windows, a conversational attribute is generated. A user-perceivable output indicative of the conversational attribute is provided.
1 . A method of providing real-time feedback of conversational attributes, the method comprising:
receiving an audio signal comprising speech;
dividing the audio signal into a plurality of sequential windows;
extracting a plurality of features from each sequential window of the audio signal;
sequentially providing each plurality of features to a trained artificial neural network, in particular a convolutional neural network, and receiving therefrom a speech attribute of the corresponding window of the audio signal, the speech attribute comprising a speaker type;
after receiving each speech attribute, and based upon that speech attribute and speech attributes of prior windows, generating a conversational attribute, wherein the conversational attribute comprises a conversational turn count (CTC) based on a change in speech attribute among that speech attribute and speech attributes of prior windows; and
providing a user-perceivable output indicative of the conversational attribute.
2 . The method of claim 1 , wherein each sequential window is about one second in length.
3 . The method of claim 1 , wherein each plurality of features comprise Mel-Frequency Cepstral Coefficients (MFCCs).
4 . The method of claim 1 , further comprising:
determining one or more environmental attributes during each of the plurality of sequential windows;
providing the one or more environmental attributes to the trained classifier with the plurality of features of the corresponding window.
5 . The method of claim 4 , wherein the environmental attributes comprises one or more of: temperature, positions, audio volume, vibration, and light level.
6 . The method of claim 1 , wherein each plurality of features comprise Mel-Frequency Cepstral Coefficients (MFCCs) and wherein providing each of the plurality of features to the trained artificial neural network comprises generating an image of the MFCCs.
7 . The method of claim 1 , wherein the speaker type is selected from: female, child, male, and non-human.
8 . The method of claim 1 , wherein the speech attribute comprises tone.
9 . The method of claim 1 , wherein the speech attribute comprises parentese.
10 . The method of claim 1 , wherein the user-perceivable output comprises a light, sounds, or vibration.
11 . The method of claim 1 , wherein providing the user-perceivable output comprises illuminating one of a plurality of lights according to whether the CTC exceeds a predetermined threshold.
12 . The method of claim 1 , wherein generating the conversational attribute comprises:
identifying child speech in one of the plurality of sequential windows; and
searching a predetermined number of prior windows for adult speech.
13 . The method of claim 1 , wherein the convolutional neural network includes exactly one convolutional layer.
14 . The method of claim 1 , wherein the convolutional neural network is quantized.
15 . A device for providing real-time feedback of conversational attributes, the device comprising:
a computing node configured to perform a method comprising:
receiving an audio signal comprising speech;
dividing the audio signal into a plurality of sequential windows;
extracting a plurality of features from each sequential window of the audio signal;
sequentially providing each plurality of features to a trained artificial neural network, in particular a convolutional neural network and receiving therefrom a speech attribute of the corresponding window of the audio signal, the speech attribute comprising a speaker type;
after receiving each speech attribute, and based upon that speech attribute and speech attributes of prior windows, generating a conversational attribute, wherein the conversational attribute comprises a conversational turn count (CTC) based on a change in speech attribute among that speech attribute and speech attributes of prior windows; and
providing a user-perceivable output indicative of the conversational attribute; and
a microphone configured to produce the audio signal;
a light, a speaker, or a vibration motor configured to produce the user-perceivable output.
16 . A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:
receiving an audio signal comprising speech;
dividing the audio signal into a plurality of sequential windows;
extracting a plurality of features from each sequential window of the audio signal;
sequentially providing each plurality of features to a trained artificial neural network in particular a convolutional neural network, and receiving therefrom a speech attribute of the corresponding window of the audio signal, the speech attribute comprising a speaker type;
after receiving each speech attribute, and based upon that speech attribute and speech attributes of prior windows, generating a conversational attribute, wherein the conversational attribute comprises a conversational turn count (CTC) based on a change in speech attribute among that speech attribute and speech attributes of prior windows; and
providing a user-perceivable output indicative of the conversational attribute.