Neural network wiring discovery

1. A method, comprising:

assigning a respective weight to each edge of a plurality of edges connecting nodes of a neural graph, wherein the neural graph represents a neural network;

designating a subset of edges from the plurality of edges based on the respective weights;

passing data through the neural graph in a forward training pass using the designated subset of edges;

determining a loss function based on results of the forward training pass;

updating parameters of the neural network and the respective weights assigned to each edge of the plurality of edges in a backwards training pass based on the loss function;

performing the steps of designating the subset of edges, passing data through the neural graph, determining the loss function, and updating parameters of the neural network and the respective weights for a plurality of iterations to train the neural network; and

storing the trained neural network for deployment on a computing device.

2. The method of claim 1 , wherein designating the subset of edges comprises:

determining an absolute value of each of the respective weights;

sorting the plurality of edges based on the absolute values of the respective weights; and

designating a predetermined number of the edges in the plurality of edges having absolute respective weight values greater than the absolute respective weight values of the remaining edges of the plurality of edges as the subset of edges.

3. The method of claim 2 , wherein a subset of edges designated during a first iteration is different than a subset of edges designated during a second iteration.

4. The method of claim 1 , wherein updating parameters of the neural network and the respective weights of the plurality of edges comprises:

determining a gradient with respect to the plurality of edges for minimizing the loss function;

passing the gradient through the designated subset of edges in the backwards training pass; and

passing the gradient to but not through remaining edges of the plurality of edges in the backwards training pass.

5. The method of claim 1 , wherein the plurality of edges comprises edges connecting each node in the neural graph to every other node in the neural graph.

6. The method of claim 1 , wherein the neural network is a convolutional neural network with each node configured to perform a convolution on input data to generate output data, the method further comprising:

passing input data having a first size through each node to generate output data having a second size smaller than the first size; and

padding the generated output data for the generated output data to have the first size.

7. The method of claim 1 , wherein the neural graph is a static graph having no looping paths.

8. The method of claim 1 , wherein the neural graph is a dynamic graph comprising at least one looping path.

9. A system, comprising:

a processor; and

a memory device containing instructions which, when executed by the processor, cause the processor to:

assign a respective weight to each edge of a plurality of edges connecting nodes of a neural graph, wherein the neural graph represents a neural network and the plurality of edges comprises edges connecting each node in the neural graph to every other node in the neural graph;

designate a subset of edges from the plurality of edges based on the respective weights;

pass data through the neural graph in a forward training pass using the designated subset of edges;

determine a loss function based on the results of the forward training pass;

update parameters of the neural network and the respective weights assigned to each edge of the plurality of edges in a backwards training pass based on the loss function;

perform the steps of designating the subset of edges, passing data through the neural graph, determining the loss function, and updating parameters of the neural network and the respective weights for a plurality of iterations to train the neural network; and

store the trained neural network for deployment on a computing device.

10. The system of claim 9 , wherein to designate the subset of edges the instructions, when executed by the processor, cause the processor to:

determine an absolute value of each of the respective weights;

sort the plurality of edges based on the absolute values of the respective weights; and

designate a predetermined number of the edges in the plurality of edges having absolute respective weight values greater than the absolute respective weight values of the remaining edges of the plurality of edges as the subset of edges.

11. The system of claim 10 , wherein a subset of edges designated during a first iteration is different than a subset of edges designated during a second iteration.

12. The system of claim 9 , wherein to update parameters of the neural network and the respective weights of the plurality of edges the instructions, when executed by the processor, cause the processor to:

determine a gradient with respect to the plurality of edges for minimizing the loss function;

pass the gradient through the designated subset of edges in the backwards training pass; and

pass the gradient to but not through remaining edges of the plurality of edges in the backwards training pass.

13. The system of claim 9 , wherein the neural network is a convolutional neural network with each node configured to perform a convolution on input data to generate output data, and

wherein the instructions, when executed by the processor, cause the processor to:

pass input data having a first size through each node to generate output data having a second size smaller than the first size; and

pad the generated output data for the generated output data to have the first size.

14. A computer program product comprising code stored in a non-transitory computer-readable storage medium, the code comprising:

code for assigning a respective weight sampled from a distribution of weights to each edge of a plurality of edges connecting nodes of a neural graph, wherein the neural graph represents a neural network;

code for designating a subset of edges from the plurality of edges based on the respective weights;

code for passing data through the neural graph in a forward training pass using the designated subset of edges;

code for determining a loss function based on results of the forward training pass;

code for updating parameters of the neural network and the respective weights assigned to each edge of the plurality of edges in a backwards training pass based on the loss function;

code for performing the steps of designating the subset of edges, passing data through the neural graph, determining the loss function, and updating parameters of the neural network and the respective weights for a plurality of iterations to train the neural network; and

code for storing the trained neural network for deployment on a computing device.

15. The computer program product of claim 14 , wherein the code for designating the subset of edges comprises:

code for determining an absolute value of each of the respective weights;

code for sorting the plurality of edges based on the absolute values of the respective weights; and

code for designating a predetermined number of the edges in the plurality of edges having absolute respective weight values greater than the absolute respective weight values of the remaining edges of the plurality of edges as the subset of edges.

16. The computer program product of claim 15 , wherein a subset of edges designated during a first iteration is different than a subset of edges designated during a second iteration.

17. The computer program product of claim 14 , wherein the code for updating parameters of the neural network and the respective weights of the plurality of edges comprises:

code for determining a gradient with respect to the plurality of edges for minimizing the loss function;

code for passing the gradient through the designated subset of edges in the backwards training pass; and

code for passing the gradient to but not through remaining edges of the plurality of edges in the backwards training pass.

18. The computer program product of claim 14 , wherein the plurality of edges comprises edges connecting each node in the neural graph to every other node in the neural graph.

19. The computer program product of claim 14 , wherein the neural network is a convolutional neural network with a plurality of nodes configured to perform a convolution on input data to generate output data, and wherein the code further comprises:

code for passing input data having a first size through each of the plurality of nodes to generate output data having a second size smaller than the first size; and

code for padding the generated output data for the generated output data to have the first size.

20. The computer program product of claim 19 , wherein the generated output data is padded with zeros.