Simultaneous measurements of gradients in optical networks
Improved training of optical neural networks is provided. In one example: 1) we choose input and target vectors, we program those into an input vector generator and a measurement unit, respectively, we turn on the optical input source power, and we monitor the electrical signal representing the cost function. 2) we can then modulate two or more controllable elements inside the optical network at different frequencies and look for the size and sign of the corresponding distinct AC variations in the measured cost function, simultaneously giving us the gradients with respect to each element.
1 . A method of training a photonic neural network, the method comprising:
providing an optical network having two or more optical inputs, two or more optical outputs and two or more control inputs, wherein control signals provided to the control inputs determine an input-output relation between the optical inputs and the optical outputs;
providing one or more predetermined input training patterns to the optical inputs of the optical network;
providing an adjustable output analyzer connected to the optical outputs of the optical network and configured to provide a cost function output;
simultaneously measuring two or more derivatives of the cost function with respect to the control signals as part of training the photonic neural network with the one or more predetermined input training patterns;
wherein the simultaneously measuring two or more derivatives of the cost function with respect to the control signals comprises dithering two or more of the control signals at two or more distinct dither frequencies and measuring corresponding distinct frequency components in the cost function output.
2 . The method of claim 1 , wherein two or more predetermined input training patterns are provided to the optical inputs of the optical network at various times, whereby the two or more derivatives of the cost function are analog time averages over the two or more predetermined input training patterns.
3 . The method of claim 1 , wherein two or more predetermined input training patterns are provided to the optical inputs of the optical network at two or more distinct wavelengths, whereby the two or more derivatives of the cost function are analog wavelength averages over the two or more predetermined input training patterns.
4 . The method of claim 1 , wherein the one or more predetermined input training patterns are provided as modulated input training patterns, whereby the corresponding distinct frequency components in the cost function output resulting from the two or more distinct dither frequencies are heterodyne shifted away from the two or more distinct dither frequencies.
5 . The method of claim 1 , further comprising adjusting the control signals to optimize the cost function with an optimization method that makes use of the two or more derivatives of the cost function with respect to the control signals, whereby the photonic neural network is trained according to the one or more predetermined input training patterns.
6 . The method of claim 1 , wherein the optical network includes two or more meshes of linear optical components connected in alternating series via one or more nonlinearity units, and wherein the control inputs include at least inputs to each of the two or more meshes of linear optical components.
7 . The method of claim 1 , wherein the optical network includes at least one optical element having a compound control input, wherein the compound control input includes a first input and a second input, wherein the first input has a lower bandwidth than the second input, and wherein a dither of the compound control input is delivered via the second input.