Multichannel audio speech classification
Examples of the present disclosure describe systems and methods for multichannel audio speech classification. In examples, an audio signal comprising multiple audio channels is received at a processing device. Each of the audio channels in the audio signal is transcoded to a predefined audio format. For each of the transcoded audio channels, an average power value is calculated for one or more data windows in the audio signal. A correlation value is calculated between the average power value for each audio channel and the combined average power value of the other audio channels in the audio signal. Each of the correlation values (or an aggregated correlation value for the audio channels) is then compared against a threshold value to determine whether the audio signal is to be classified as a speech-based communication. Based on the classification, an action associated with the audio signal may be performed.
1 . A system comprising:
a processor; and
memory comprising computer executable instructions that, when executed, perform operations comprising:
identifying an audio signal comprising a first audio channel and a second audio channel;
calculating a first average power value for a first data window in the first audio channel;
calculating a second average power value for a second data window in the second audio channel;
determining a correlation value for the first data window in the first audio channel and the second data window in the second audio channel based on the first average power value and the second average power value;
generating, based on the correlation value, a classification of the audio signal as one of:
multi-speaker speech;
single speaker speech;
speech comprising non-speech audio elements; or
non-speech; and
performing an action for the audio signal based on the classification, wherein the action comprises one of: audio transcription, speaker diarization, or acoustic event detection.
2 . The system of claim 1 , wherein identifying the audio signal comprises:
determining the audio signal comprises at least the first audio channel and the second audio channel;
transcoding the first audio channel into a first transcoded audio channel; and
transcoding the second audio channel into a second transcoded audio channel.
3 . The system of claim 2 , wherein the first transcoded audio channel and the second transcoded audio channel are in a same audio format having a specific bit rate.
4 . The system of claim 1 , wherein calculating the first average power value for the first data window comprises:
identifying at least one data window in the first audio channel based on a set of parameters including at least one of stride length or window size, the at least one data window including the first data window.
5 . The system of claim 4 , wherein the stride length defines a number of audio signal data values between data windows of the first audio channel.
6 . The system of claim 4 , wherein the window size defines a number of audio signal data values within a data window of the first audio channel.
7 . The system of claim 4 , wherein the set of parameters is configured manually using a user interface provided by the system, the user interface comprising interface elements enabling a user to define parameters and parameter values of the set of parameters.
8 . The system of claim 4 , wherein the set of parameters is configured automatically by the system based on at least one of:
a length of the first audio channel;
a data size of the first audio channel; or
an audio format of the first audio channel.
9 . The system of claim 1 , wherein the first data window and the second data window represent a same segment of time within the first audio channel and the second audio channel.
10 . The system of claim 1 , wherein calculating the first average power value for the first data window comprises:
squaring an amplitude of each audio signal data value in the first data window to generate squared amplitudes; and
averaging the squared amplitudes.
11 . The system of claim 1 , wherein the correlation value identifies:
a positive correlation between the first data window and the second data window;
a negative correlation between the first data window and the second data window; or
a neutral correlation between the first data window and the second data window.
12 . The system of claim 1 , wherein generating the classification for the audio signal comprises comparing the correlation value to one or more thresholds, each of the one or more thresholds representing a classification of speech.
13 . A method comprising:
calculating a first average power value for a first data window in a first audio channel of an audio signal;
calculating a second average power value for a second data window in a second audio channel of the audio signal;
determining a correlation value for the first data window in the first audio channel and the second data window in the second audio channel based on the first average power value and the second average power value;
generating a classification of the audio signal as a particular speech category by comparing the correlation value to at least one threshold value associated with the particular speech category; and
performing an action for the audio signal based on the classification, wherein the action comprises one of: audio transcription, speaker diarization, or acoustic event detection.
14 . The method of claim 13 , wherein the particular speech category corresponds to:
multi-speaker speech;
single speaker speech;
speech comprising non-speech audio elements; or
non-speech.
15 . The method of claim 13 , further comprising:
providing an indication of the particular speech category to a user or a device.
16 . The method of claim 15 , further comprising:
providing at least one confidence score for the particular speech category to the user or the device, the at least one confidence score indicating a probability that the particular speech category is accurate for the audio signal.
17 . The method of claim 15 , wherein providing the indication of the particular speech category includes providing the correlation value to the user or the device.
18 . A device comprising:
a processor; and
memory comprising computer executable instructions that, when executed, perform operations comprising:
calculating a first average power value for a first data window in a first audio channel of an audio signal;
calculating a second average power value for a second data window in a second audio channel of the audio signal;
determining a correlation value for the first data window in the first audio channel and the second data window in the second audio channel based on the first average power value and the second average power value;
identifying a particular speech category for the audio signal by comparing the correlation value to a threshold value associated with the particular speech category; and
performing an action for the audio signal based on the particular speech category, wherein the action comprises one of: audio transcription, speaker diarization, or acoustic event detection.
19 . The device of claim 18 , wherein the first data window and the second data window represent a same segment of time within the first audio channel and the second audio channel.
20 . The device of claim 18 , wherein calculating the first average power value for the first data window comprises:
squaring an amplitude of each audio signal data value in the first data window to generate squared amplitudes; and
averaging the squared amplitudes.