System and method for automated observation and analysis of instructional discourse
A method of analyzing instructor discourse includes recording an audio signal representing speech of the instructor during a class session, converting the audio signal to a session transcript comprising speech data for the session using an automatic speech recognition tool and segmenting the transcript into utterances, extracting a set of features from the session transcript, filtering student talk out from the utterances, analyzing a first subset of the features to produce a number of local context predictions for each utterance of the session transcript, analyzing a second subset of the features to produce a number of global context predictions for the session transcript, and combining a subset of the number of local context predictions and the number of global context predictions into a classification that attends to differential reliability.
1. An automated method of analyzing instructor discourse, comprising:
recording, with an audio recording device, an audio signal representing speech occurring during a session of discourse between an instructor and students;
transmitting the audio signal, with a signal transmitter, to a computing system including audio processing software;
converting the audio signal, with the audio processing software of the computing system, to an electronic session transcript comprising speech data for the session using an automatic speech recognition tool of the audio processing software, wherein the electronic session transcript includes a plurality of utterances and wherein the audio processing software processes the electronic session transcript using natural language processing instructions to identify and segment the plurality of utterances;
extracting, with the audio processing software of the computing system, a set of talk features from the electronic session transcript;
filtering, with the audio processing software of the computing system, student talk out from the plurality of utterances to produce a plurality of instructor utterances by analyzing the audio signal;
concurrently analyzing, with the audio processing software of the computing system, (i) each instructor utterance using a first number of the talk features to produce a number of local context predictions for the instructor utterances and (ii) a merger of all of the instructor utterances using a second number of the talk features to produce a number of global context predictions for the electronic session transcript, each local context prediction pertaining to the presence or absence of a number of discourse variables in each instructor utterance and each global context prediction pertaining to a proportional occurrence of each discourse variable over the entirety of the session; and
combining, with the audio processing software of the computing system, a subset of the number of local context predictions and the number of global context predictions into a classification,
wherein, for a given instructor utterance in the plurality of instructor utterances, the classification signifies that a differentiation has been made between the following discourse variable types in classifying the given utterance: instructional talk, questions, authentic questions, elaborated evaluation, high cognitive level, uptake, goal specificity, and ELA terms.
2. The method according to claim 1 , further comprising producing an estimate of a reliability of the classification based on a number of input features.
3. The method according to claim 2 , wherein the number of input features include one or more of statistics relating to a transcription confidence of the converting by the automatic speech recognition tool, a direct measure of a number of acoustic-prosodic features of audio signal, a number of utterance level word features of the session transcript, a number of session level word/text features of the session transcript, utterance- and session-level contextual embeddings, and estimates of a number of the talk features of the session transcript.
4. The method according to claim 1 , wherein the combining comprises scaling the subset of the number of local context predictions and the number of global context predictions to a human range, and combining the scaled subset of the number of local context predictions and number of global context predictions into the classification.
5. The method according to claim 1 , wherein the analyzing each instructor utterance using the first number of the talk features to produce the number of local context predictions is performed by a machine learning technique implemented in a trained machine learning model included in the audio processing software.
6. The method according to claim 5 , wherein the machine learning technique comprises a machine learning classifier technique.
7. The method according to claim 6 , wherein the machine learning classifier technique comprises a random forest classifier.
8. The method according to claim 1 , wherein the analyzing the merger of all the instructor utterances using the second number of the talk features to produce the number of global context predictions is performed by a machine learning technique implemented in a trained machine learning model included in the audio processing software.
9. The method according to claim 8 , wherein the machine learning technique comprises a machine learning regression technique.
10. The method according to claim 9 , wherein the machine learning regression technique comprises a random forest regression.
11. The method according to claim 1 , wherein the set of features comprises acoustic-prosodic features, context features, linguistic features, n-grams, and contextual semantics.
12. The method according to claim 11 , wherein the first number of the features includes the acoustic-prosodic features, the context features, the linguistic features, the n-grams, and the contextual semantics, and wherein the second number of the features includes only the n-grams.
13. A system for analyzing instructor discourse, comprising:
audio processing software stored in a non-transitory computer readable medium and executable by a computing system, the audio processing software including:
an automatic speech recognition component configured for receiving an audio signal representing speech occurring during a session of discourse between an instructor and students recorded by an audio recording device and transmitted to the computing system, the automatic speech recognition component being configured for converting the audio signal to an electronic session transcript comprising speech data for the session using an automatic speech recognition tool, wherein the electronic session transcript includes a plurality of utterances and wherein the automatic speech recognition component is configured to process the electronic session transcript using natural language processing instructions to identify and segment the plurality of utterances; and
an automated analysis component configured for: (i) extracting a set of talk features from the electronic session transcript, (ii) filtering student talk out from the plurality of utterances to produce a plurality of instructor utterances by analyzing the audio signal, (iii) concurrently analyzing (a) each instructor utterance using a first number of the talk features to produce a number of local context predictions for the instructor utterances and (b) a merger of all of the instructor utterances using a second number of the talk features to produce a number of global context predictions for the electronic session transcript, wherein each local context prediction pertains to the presence or absence of a number of discourse variables in each instructor utterance and each global context prediction pertains to a proportional occurrence of each discourse variable over the entirety of the session, and (iv) combining a subset of the number of local context predictions and the number of global context predictions into a classification,
wherein, for a given utterance in the plurality of utterances, the classification signifies that a differentiation has been made between the following discourse variable types in classifying the given utterance: instructional talk, questions, authentic questions, elaborated evaluation, high cognitive level, uptake, goal specificity, and ELA terms.
14. The system according to claim 13 , wherein the analysis component is further configured for producing an estimate of a reliability of the classification based on a number of input features.
15. The system according to claim 14 , wherein the number of input features include one or more of statistics relating to a transcription confidence of the converting by the automatic speech recognition tool, a direct measure of a number of acoustic-prosodic features of audio signal, a number of utterance level word features of the session transcript, a number of session level word/text features of the session transcript, utterance- and session-level contextual embeddings, and estimates of a number of the talk features of the session transcript.
16. The system according to claim 13 , wherein the combining comprises scaling the subset of the number of local context predictions and the number of global context predictions to a human range, and combining the scaled subset of the number of local context predictions and number of global context predictions into the classification.
17. The system according to claim 13 , wherein the analyzing each instructor utterance using the first number of the talk features to produce the number of local context predictions is performed by a machine learning technique implemented in a trained machine learning model included in the audio processing software.
18. The system according to claim 17 , wherein the machine learning technique comprises a machine learning classifier technique.
19. The system according to claim 18 , wherein the machine learning classifier technique comprises a random forest classifier.
20. The system according to claim 13 , wherein the analyzing the merger of all of the instructor utterances using the second number of the talk features to produce the number of global context predictions is performed by a machine learning technique implemented in a trained machine learning model included in the audio processing software.
21. The system according to claim 20 , wherein the machine learning technique comprises a machine learning regression technique.
22. The system according to claim 21 , wherein the machine learning regression technique comprises a random forest regression.
23. The system according to claim 13 , wherein the set of features comprises acoustic-prosodic features, context features, linguistic features, n-grams, and contextual semantics.
24. The system according to claim 23 , wherein the first number of the talk features includes the acoustic-prosodic features, the context features, the linguistic features, the n-grams, and the contextual semantics, and wherein the second number of the talk features includes only the n-grams.
25. The method according to claim 1 , wherein the extracting, filtering and analyzing are performed a trained machine learning model included in the audio processing software.
26. The system according to claim 13 , wherein the extracting, filtering and analyzing are performed a trained machine learning model included in the audio processing software.