Neural modeler of audio systems
A process is provided for training a neural network that digitally models an audio system. A sound source is utilized to electrically couple a test signal into an input of a reference audio system. The output of the reference audio system is collected into an audio interface coupled to a computer. A neural network is then trained using the test signal and the captured information to derive a set of weight vectors with appropriate values such that the overall output of the neural network converges towards an output representative of the reference audio system, and a signal in the time domain from a musical instrument is processed through the trained neural network with a latency under 20 milliseconds. A graphical user interface then outputs a graphical representation of the trained neural network, where the graphical representation visually displays at least one virtual control for interaction by a user.
1 . A process for training a neural network that digitally models an audio system, comprising:
utilizing a sound source to electrically couple a test signal into an input of a reference audio system;
collecting an output of the reference audio system into an audio interface coupled to a computer to store captured information;
training a neural network using the test signal and the captured information to derive a set of weight vectors with appropriate values such that:
the overall output of the neural network converges towards an output representative of the reference audio system; and
a digital signal representing an analog musical instrument signal is processed through the trained neural network in the time domain with an algorithmic latency under 20 milliseconds;
outputting to a graphical user interface, a graphical representation of the trained neural network, the graphical representation visually displaying at least one virtual control; and
enabling a user to interact with the virtual control of the graphical representation of the trained neural network via the graphical user interface to define a virtualization of the reference audio system.
2 . The process of claim 1 , wherein the training is carried out on the computer.
3 . The process of claim 1 , wherein the training is carried out on a separate, remote computer.
4 . The process of claim 1 , wherein training the neural network further comprises:
setting a stopping condition that determines when training ends by performing at least one of:
setting a user-initiated stopping condition; or
processing a number of iterations of the training data as the stopping condition.
5 . The process of claim 1 , wherein training the neural network further comprises:
setting a stopping condition that determines when training ends by processing a perceptual loss function where the perceptual loss function serves as an indicator of the stopping condition.
6 . The process of claim 1 , wherein:
outputting to the graphical user interface, the graphical representation of the trained neural network further comprises outputting to the graphical user interface, a graphical representation of an effects processor that is not within a native capability of the reference audio system.
7 . The process of claim 6 , wherein:
outputting to the graphical user interface, the graphical representation of the effects processor that is not within the native capability of the reference audio system comprises outputting to the graphical user interface, a graphical representation of an equalizer for equalization, wherein the equalizer is not part of the reference audio system.
8 . The process of claim 6 , wherein:
outputting to the graphical user interface, the graphical representation of the effects processor that is not within the native capability of the reference audio system comprises outputting to the graphical user interface, a graphical representation of a dynamics processor, wherein the dynamics processor is not part of the reference audio system.
9 . The process of claim 1 , wherein:
outputting to the graphical user interface, the graphical representation of the effects processor that is not within the native capability of the reference audio system comprises outputting to the graphical user interface, a graphical representation of a time-based processor, wherein the time-based processor is not part of the reference audio system.
10 . A process for training a neural network that digitally models an audio system, comprising:
utilizing a sound source to electrically couple a test signal into an input of a reference audio system;
collecting an output of the reference audio system into an audio interface coupled to a computer to store captured information;
training a neural network using the test signal and the captured information to derive a set of weight vectors with appropriate values such that the overall output of the neural network converges towards an output representative of the reference audio system, and a signal in the time domain from a musical instrument is processed through the trained neural network with a latency under 20 milliseconds;
outputting to a graphical user interface, a graphical representation of the trained neural network, the graphical representation visually displaying at least one virtual control; and
enabling a user to interact with the virtual control of the graphical representation of the trained neural network via the graphical user interface to define a virtualization of the reference audio system;
wherein training the neural network further comprises modeling a non-linear behavior of the reference audio system, modeling a first linear aspect of the reference audio system, and modeling a second linear aspect of the reference audio system.
11 . The process of claim 1 , wherein:
outputting to the graphical user interface, the graphical representation of the trained neural network further comprises outputting to the graphical user interface, a graphical representation of at least one of an emulation of a speaker and an emulation of a speaker cabinet, wherein the emulation is not part of the reference audio system.
12 . The process of claim 1 , wherein:
training the neural network models a non-linear behavior and a linear aspect of the reference audio system.
13 . The process of claim 1 , wherein:
training the neural network further comprises creating a model file that includes sufficient data such that that when read out and processed by a modeling audio system, a functioning model of the reference audio system is realized; wherein:
the virtualization comprises a framework that enables the modeling audio system to model the reference audio system by loading the model file into the modeling audio system.
14 . A process for creating digital audio systems, comprising:
utilizing a sound source to electrically couple a test signal into an input of a reference audio system;
collecting an output of the reference audio system into an audio interface coupled to a computer to store captured information;
training a neural network using the test signal and the captured information to derive a set of weight vectors with appropriate values such that the overall output of the neural network converges towards an output representative of the reference audio system, wherein the training digitally models a non-linear behavior of the reference audio system, models a first linear aspect of the reference audio system, and models a second linear aspect of the reference audio system, the training carried out by repeatedly performing operations comprising:
predicting by the neural network, a model output based upon an input, where the output approximates an expected output of the reference audio system;
computing an error in the prediction; and
adjusting the weight vectors to minimize the computed error;
outputting a neural network model file upon training;
outputting to a graphical user interface, a graphical representation of the trained neural network in the neural network model file, the graphical representation visually displaying at least one virtual control; and
enabling a user to interact with the virtual control of the graphical representation of the trained neural network via the graphical user interface to define a virtualization of the reference audio system.
15 . The process of claim 14 , wherein predicting by the neural network, the model output comprises carrying out the prediction in the time domain.
16 . The process of claim 14 , wherein:
modeling the non-linear behavior of the reference audio system, modeling the first linear aspect of the reference audio system, and modeling the second linear aspect of the reference audio system are arranged in series, parallel, or a combination thereof.
17 . The process of claim 16 further comprising modeling a temporal dependency of the reference audio system in addition to, or in lieu of modeling the first linear aspect of the reference audio system.
18 . The process of claim 14 , wherein the training further comprises:
applying a perceptual loss function to the neural network based upon a determined psychoacoustic property, wherein the perceptual loss function is applied in the frequency domain; and
adjusting the neural network responsive to the output of the perceptual loss function.
19 . The process of claim 18 , wherein applying the perceptual loss function to the neural network comprises establishing a loudness threshold of hearing for each of multiple frequency bins such that a signal below the loudness threshold is not optimized further, wherein:
for each frequency bin, a loudness threshold is independently set under which a signal is not optimized further in order to optimize further that particular frequency bin.
20 . The process of claim 18 , wherein applying the perceptual loss function to the neural network comprises:
implementing frequency masking such that a frequency component is not further processed if a computed error is below a masking threshold, where the masking threshold is based upon a target signal.