Audio datasets generation for videoconferencing

1 . A method for generating a synthetic audio signal, the method comprising:

recording, by an audio recording system, a set of raw audio signals, the set of raw audio signals comprising audio signals recorded by a plurality of microphones at a plurality of distances from an audio source with a plurality of orientations;

accessing, by an audio dataset generation system, a virtual meeting room setup, the virtual meeting room setup specifying microphones used in a virtual meeting room, locations of speakers and the microphones, and orientations of the speakers; and

generating, by the audio dataset generation system, a synthetic audio signal for each of the speakers specified in the virtual meeting room setup comprising combining audio signals selected from the set of raw audio signals according to the virtual meeting room setup.

2 . The method of claim 1 , wherein recording the set of raw audio signals comprises:

controlling the audio source to move to a first distance from the plurality of microphones;

controlling the audio source to face to a first orientation of the plurality of orientations;

recording audio signals played on the audio source using the plurality of microphones to generate a first subset of raw audio signals of the set of raw audio signals corresponding to the first distance and the first orientation;

controlling the audio source to have a second orientation;

recording audio signals played on the audio source using the plurality of microphones to generate a second subset of raw audio signals of the set of raw audio signals corresponding to the first distance and the second orientation;

controlling the audio source to move to a second distance from the plurality of microphones;

controlling the audio source to have the first orientation;

recording audio signals played on the audio source using the plurality of microphones to generate a third subset of raw audio signals of the set of raw audio signals corresponding to the second distance and the first orientation;

controlling the audio source to have the second orientation; and

recording audio signals played on the audio source using the plurality of microphones to generate a second subset of raw audio signals of the set of raw audio signals corresponding to the second distance and the second orientation.

3 . The method of claim 2 , wherein the audio source is a head speaker, and the plurality of orientations comprises a 0-degree orientation, a 90-degree orientation, a 180-degree orientation, and a 270-degree orientation.

4 . The method of claim 1 , wherein generating the synthetic audio signal comprises:

constructing, according to the virtual meeting room setup, a scenario matrix describing relationships between each of the speakers and each of the microphones;

extracting audio signals from the set of raw audio signals based on the scenario matrix; and

combining the extracted audio signals to generate respective synthetic audio signals for the microphones.

5 . The method of claim 4 , wherein the relationships between each of the speakers and each of the microphones comprise a distance between the speaker and the microphone and an orientation of the speaker relative to the microphone.

6 . The method of claim 4 , wherein generating the synthetic audio signal further comprises:

generating a quality score matrix for the extracted audio signals, each value in the quality score matrix indicating a quality of a corresponding extracted audio signal that corresponds to a specific speaker as recorded by a particular microphone;

generating ground truth labels for the speakers based on the quality score matrix identifying a highest-quality microphone for each speaker; and

generating active speaker labels indicating active speakers of respective segments of the synthetic audio signal.

7 . The method of claim 1 , further comprising pre-processing the set of raw audio signals by at least:

removing a silent portion of each of the set of raw audio signals; and

aligning the set of raw audio signals.

8 . The method of claim 1 , wherein each audio signal in the set of raw audio signals comprise human speeches.

9 . The method of claim 1 , further comprising causing the synthetic audio signal to be used as training data or testing data for an audio processing model.

10 . A system comprising:

an audio recording system, comprising:

a plurality of microphones;

an audio source; and

a computing device configured to:

record a set of raw audio signals, the set of raw audio signals comprising audio signals recorded by a plurality of microphones at a plurality of distances from an audio source with a plurality of orientations; and

an audio dataset generation system, comprising:

a non-transitory computer-readable medium; and

a processor communicatively coupled to the non-transitory computer-readable medium, the processor configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

access a virtual meeting room setup, the virtual meeting room setup specifying microphones used in a virtual meeting room, locations of speakers and the microphones, and orientations of the speakers; and

generate a synthetic audio signal for each of the speakers specified in the virtual meeting room setup comprising combining audio signals selected from the set of raw audio signals according to the virtual meeting room setup.

11 . The system of claim 10 , wherein recording the set of raw audio signals comprises:

controlling the audio source to move to a first distance from the plurality of microphones;

controlling the audio source to face to a first orientation of the plurality of orientations;

recording audio signals played on the audio source using the plurality of microphones to generate a first subset of raw audio signals of the set of raw audio signals corresponding to the first distance and the first orientation;

controlling the audio source to have a second orientation;

recording audio signals played on the audio source using the plurality of microphones to generate a second subset of raw audio signals of the set of raw audio signals corresponding to the first distance and the second orientation;

controlling the audio source to move to a second distance from the plurality of microphones;

controlling the audio source to have the first orientation;

recording audio signals played on the audio source using the plurality of microphones to generate a third subset of raw audio signals of the set of raw audio signals corresponding to the second distance and the first orientation;

controlling the audio source to have the second orientation; and

recording audio signals played on the audio source using the plurality of microphones to generate a second subset of raw audio signals of the set of raw audio signals corresponding to the second distance and the second orientation.

12 . The system of claim 10 , wherein generating the synthetic audio signal comprises:

constructing, according to the virtual meeting room setup, a scenario matrix describing relationships between each of the speakers and each of the microphones;

extracting audio signals from the set of raw audio signals based on the scenario matrix; and

combining the extracted audio signals to generate respective synthetic audio signals for the microphones.

13 . The system of claim 12 , wherein the relationships between each of the speakers and each of the microphones comprise a distance between the speaker and the microphone and an orientation of the speaker relative to the microphone.

14 . The system of claim 12 , wherein generating the synthetic audio signal further comprises:

generating a quality score matrix for the extracted audio signals, each value in the quality score matrix indicating a quality of a corresponding extracted audio signal that corresponds to a specific speaker as recorded by a particular microphone;

generating ground truth labels for the speakers based on the quality score matrix identifying a highest-quality microphone for each speaker; and

generating active speaker labels indicating active speakers of respective segments of the synthetic audio signal.

15 . The system of claim 10 , wherein the processor is further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to pre-process the set of raw audio signals by at least:

removing a silent portion of each of the set of raw audio signals; and

aligning the set of raw audio signals.

16 . A non-transitory computer-readable medium comprising processor-executable instructions configured to cause one or more processors to:

access a virtual meeting room setup, the virtual meeting room setup specifying microphones used in a virtual meeting room, locations of speakers and the microphones, and orientations of the speakers; and

generate a synthetic audio signal for each of the speakers specified in the virtual meeting room setup comprising combining audio signals selected from a set of raw audio signals according to the virtual meeting room setup, wherein the set of raw audio signals comprise audio signals recorded by a plurality of microphones at a plurality of distances from an audio source with a plurality of orientations.

17 . The non-transitory computer-readable medium of claim 16 , wherein recording the set of raw audio signals comprises:

controlling the audio source to move to a first distance from the plurality of microphones;

controlling the audio source to face to a first orientation of the plurality of orientations;

recording audio signals played on the audio source using the plurality of microphones to generate a first subset of raw audio signals of the set of raw audio signals corresponding to the first distance and the first orientation;

controlling the audio source to have a second orientation;

recording audio signals played on the audio source using the plurality of microphones to generate a second subset of raw audio signals of the set of raw audio signals corresponding to the first distance and the second orientation;

controlling the audio source to move to a second distance from the plurality of microphones;

controlling the audio source to have the first orientation;

recording audio signals played on the audio source using the plurality of microphones to generate a third subset of raw audio signals of the set of raw audio signals corresponding to the second distance and the first orientation;

controlling the audio source to have the second orientation; and

recording audio signals played on the audio source using the plurality of microphones to generate a second subset of raw audio signals of the set of raw audio signals corresponding to the second distance and the second orientation.

18 . The non-transitory computer-readable medium of claim 16 , wherein generating the synthetic audio signal comprises:

constructing, according to the virtual meeting room setup, a scenario matrix describing relationships between each of the speakers and each of the microphones;

extracting audio signals from the set of raw audio signals based on the scenario matrix; and

combining the extracted audio signals to generate respective synthetic audio signals for the microphones.

19 . The non-transitory computer-readable medium of claim 18 , wherein the relationships between each of the speakers and each of the microphones comprise a distance between the speaker and the microphone and an orientation of the speaker relative to the microphone.

20 . The non-transitory computer-readable medium of claim 18 , wherein generating the synthetic audio signal further comprises:

generating a quality score matrix for the extracted audio signals, each value in the quality score matrix indicating a quality of a corresponding extracted audio signal that corresponds to a specific speaker as recorded by a particular microphone;

generating ground truth labels for the speakers based on the quality score matrix identifying a highest-quality microphone for each speaker; and

generating active speaker labels indicating active speakers of respective segments of the synthetic audio signal.