System and method of a monotone operator neural network
A system for training a neural work that includes an input interface for accessing input data for the neural network and a processor in communication with the input interface. The processor is programmed to receive input at the neural network and output a trained neural networking utilizing a forward prorogation and a backward propagation, wherein the forward propagation includes utilizing a root-finding procedure to identify a fixed point associated with one or more parameters of the neural network, wherein the backward propagation includes identifying a derivative of a loss associated with the parameters of the network.
1. A computer-implemented method for predicting a classification and training a neural network, comprising:
receiving input at the neural network;
classifying the input of the neural networking utilizing a forward propagation, wherein the forward propagation includes utilizing a root-finding procedure to identify a fixed point associated with one or more parameters of the neural network;
computing a vector by utilizing a linear transformation of the fixed point;
outputting the classification of the input in response to a maximum element of the vector; and
training the neural network utilizing a backward propagation, wherein the backward propagation includes identifying a derivative of a loss associated with the parameters of the neural network.
2. The computer-implemented method of claim 1 , wherein performing backwards propagation includes utilizing a Forward-Backward splitting method.
3. The computer-implemented method of claim 1 , wherein performing backwards propagation includes utilizing a Peaceman-Rachford splitting method.
4. The computer-implemented method of claim 1 , wherein the input includes image data.
5. The computer-implement method of claim 1 , wherein the forward propagation includes utilizing a Forward-Backward splitting method to identify the fixed point.
6. The computer-implement method of claim 1 , wherein the forward propagation includes utilizing the forward propagation until a convergence.
7. The computer-implemented method of claim 1 , wherein the forward propagation includes utilizing a Peaceman-Rachford splitting method to identify the fixed point.
8. The computer-implemented method of claim 1 , wherein the backward propagation includes multiplying a Jacobian of a partial derivative of the fixed point of a nonlinear function.
9. The computer-implemented method of claim 1 , wherein the backward propagation includes utilization of a hyperparameter.
10. A computer-implemented method for a classification of input to a neural network, comprising:
receiving input at the neural network;
identifying a classification of the input at the neural networking utilizing a forward propagation, wherein the forward propagation includes utilizing a root-finding procedure to identify a fixed point associated with a parameter of the neural network;
computing a prediction vector by utilizing a linear transformation of the fixed point; and
outputting the classification of the input in response to a maximum value of the prediction vector.
11. The computer-implemented method of claim 10 , wherein performing the forward propagation includes utilizing a Forward-Backward splitting method.
12. The computer-implemented method of claim 10 , wherein performing the forward propagation includes utilizing a Peaceman-Rachford splitting method.
13. The computer-implemented method of claim 10 , wherein the input at the neural network includes image data received from one or more cameras.
14. The computer-implemented method of claim 10 , wherein the fixed point is identified utilizing a non-linear function.
15. A system for training a neural network, comprising:
an input interface for accessing input data for the neural network; and
a processor in communication with the input interface, the processor programmed to:
receive the input data at the neural network; and
output a trained neural network utilizing a forward propagation and a backward propagation, wherein the forward propagation includes utilizing a root-finding procedure to identify a fixed point associated with one or more parameters of the neural network, wherein the backward propagation includes identifying a derivative of a loss associated with the parameters of the network.
16. The system of claim 15 , wherein performing the backward propagation includes utilizing a Forward-Backward splitting method.
17. The system of claim 15 , wherein performing the backward propagation includes utilizing a Peaceman-Rachford splitting method.
18. The system of claim 15 , wherein the processor is further programmed to adjust the parameters of the neural network utilizing a gradient associated with the parameters.
19. The system of claim 15 , wherein the processor is further programmed to run the forward propagation for each minibatch in response to an index.
20. The system of claim 15 , wherein the neural network is a convolutional neural network.