TRAINING A NODAL NETWORK SO THAT A FIRST NODE HAS A HIGH MAGNITUDE CORRELATION WITH THE PARTIAL DERIVATIVE OF THE OBJECTIVE WITH RESPECT TO A SECOND NODE
A system and method for controlling a nodal network. The method includes estimating an effect on the objective caused by the existence or non-existence of a direct connection between a pair of nodes and changing a structure of the nodal network based at least in part on the estimate of the effect. A nodal network includes a partially ordered set, a weighted directed acyclic graph, an artificial neural network, and/or a layered feed-forward neural network.
1 - 101 . (canceled)
102 . A method comprising:
training, by a computer system, a first nodal network with a set of training data items such that a magnitude of a correlation of (i) an activation vector of an output node A of the first nodal network to (ii) a derivative vector for a node B of a second nodal network is above a threshold, wherein the activation vector comprises an element for each of the training data items in the set of training data items, and wherein the derivative vector comprises a partial derivative of an objective for the second nodal network with respect to input to the node B for each of the training data items in the set of training data items.
103 . The method of claim 102 , further comprising computing, by the computer system, the derivative vector for the node B.
104 . The method of claim 102 , further comprising, incorporating, by the computer system, the first network into the second network to make a combined network such that node B covers node A in the combined network.
105 . The method of claim 1 . 04 , further comprising, after incorporating the first network into the second network, training, by the computer, the combined network.
106 . The method of claim 102 , wherein the nodal network comprises a self-organizing partially order nodal network.
107 . A computer system comprising:
one or more processor cores; and
a memory in communication with the one or more processor cores, wherein the memory stores software that, when executed by the one or more processor cores, cause the one or more processor cores to:
train a first nodal network with a set of training data items such that a magnitude of a correlation of (i) an activation vector of an output node A of the first nodal network to (ii) a derivative vector for a node B of a second nodal network is above a threshold, wherein the activation vector comprises an element for each of the training data items in the set of training data items, and wherein the derivative vector comprises a partial derivative of an objective for the second nodal network with respect to input to the node B for each of the training data items in the set of training data items.
108 . The computer system of claim 107 , wherein the memory stores further software that, when executed by the one or more processors, cause the one or more processors to compute the derivative vector for the node B.
109 . The computer system of claim 112 , wherein the memory stores further software that, when executed by the one or more processors, cause the one or more processors to incorporate the first network into the second network to make a combined network such that node B covers node A in the combined network.
110 . The computer system of claim 109 , wherein the memory stores further software that, when executed by the one or more processors, cause the one or more processors to, after incorporating the first network into the second network, train the combined network.
111 . The computer system of claim 107 , wherein the nodal network comprises a self-organizing partially order nodal network.