Generating and managing deep tensor neural networks
Techniques for generating and managing, including simulating and training, deep tensor neural networks are presented. A deep tensor neural network comprises a graph of nodes connected via weighted edges. A network management component (NMC) extracts features from tensor-formatted input data based on tensor-formatted parameters. NMC evolves tensor-formatted input data based on a defined tensor-tensor layer evolution rule, the network generating output data based on evolution of the tensor-formatted input data. The network is activated by non-linear activation functions, wherein the weighted edges and non-linear activation functions operate, based on tensor-tensor functions, to evolve tensor-formatted input data. NMC trains the network based on tensor-formatted training data, comparing output training data output from the network to simulated output data, based on a defined loss function, to determine an update. NMC updates the network, including weight and bias parameters, based on the update, by application of tensor-tensor operations.
1. A system, comprising:
a memory that stores computer-executable components; and
a processor, operatively coupled to the memory, that executes computer-executable components, the computer-executable components comprising:
a network component that generates a neural network comprising a graph of nodes connected via weighted edges; and
a network management component that:
in response to the neural network receiving input data and parameters, formats the input data and the parameters to generate tensor-formatted input data and tensor-formatted parameters, and wherein at least a portion of the input data is multi-dimensional data,
extracts a first portion of features from the tensor-formatted input data in parallel with extraction of a second portion of the features from the tensor-formatted input data, and
trains the neural network using a Hamiltonian forward propagation process to evolve the tensor-formatted input data based on a defined tensor-tensor layer evolution rule, wherein the neural network generates output data based on the evolution of the tensor-formatted input data, and wherein the Hamiltonian forward propagation process comprises a multiplication of a circulant function of a first antisymmetric matrix and an unfold function of a second matrix.
2. The system of claim 1 , wherein the neural network is a deep tensor neural network, wherein the deep tensor neural network is activated by a set of non-linear activation functions, and wherein the weighted edges and the set of non-linear activation functions operate, based on tensor-tensor functions, to evolve the tensor-formatted input data.
3. The system of claim 1 , wherein the network management component applies weights to edges to generate the weighted edges, wherein the weights are applied to the edges via a tensor-tensor product or a multiply type of the tensor-tensor product.
4. The system of claim 1 , wherein the network management component receives a portion of the parameters via an interface from a user, and wherein the portion of the parameters is selected from a group of defined parameters consisting of a defined constant value, a defined network structure, and a non-linear activation function.
5. The system of claim 1 , wherein a portion of the tensor-formatted input data is tensor-formatted training data.
6. The system of claim 1 , wherein a portion of the output data is output training data derived from inputting tensor-formatted training data to the neural network, and wherein the network management component compares the output training data to simulated output data, based on a defined loss function, to determine an update to perform on the neural network to improve operation of the neural network.
7. The system of claim 6 , wherein the network management component updates the neural network, based on the update, by application of a set of tensor-tensor operations via back-propagation.
8. The system of claim 7 , wherein, as part of performing the update, the network management component updates a weight of a weighted edge of the weighted edges and updates a parameter of a non-linear activation function of the neural network.
9. A computer-implemented method, comprising:
generating, by a system operatively coupled to a processor, a neural network comprising a graph of nodes connected via weighted edges;
in response to the neural network receiving input data and parameters, formatting, by the system, the input data and the parameters to generate tensor-formatted input data and tensor-formatted parameters, and wherein at least a portion of the input data is multi-dimensional data;
extracting, by the system, a first portion of features from the tensor-formatted input data in parallel with extraction of a second portion of the features from the tensor-formatted input data; and
training, by the system, the neural network using a Hamiltonian forward propagation process to evolve the tensor-formatted input data based on a defined tensor-tensor layer evolution rule, wherein the neural network generates output data based on the evolving of the tensor-formatted input data, wherein the Hamiltonian forward propagation process comprises a multiplication of a circulant function of a first antisymmetric and an unfold function of a second matrix.
10. The computer-implemented method of claim 9 , further comprising:
activating, by the system, the neural network based on a set of non-linear activation functions, wherein the weighted edges and the set of non-linear activation functions operate, based on tensor-tensor functions, to perform the evolving of the tensor-formatted input data.
11. The computer-implemented method of claim 9 , further comprising:
applying, by the system, weights to edges, based on a tensor-tensor product or a multiply type of the tensor-tensor product, to generate the weighted edges.
12. The computer-implemented method of claim 9 , wherein a portion of the tensor-formatted input data is tensor-formatted training data, wherein a portion of the output data is output training data, and further comprising:
applying, by the system, the tensor-formatted training data to the neural network to generate the output training data; and
comparing, by the system, the output training data to simulated output data, based on a defined loss function, to determine an update to perform on the neural network to enhance operation of the neural network.
13. The computer-implemented method of claim 12 , further comprising:
modifying, by the system, the neural network, based on the update, by applying a set of tensor-tensor operations via back-propagation, wherein the modifying the neural network comprises modifying a weight value of a weighted edge of the weighted edges, and modifying a parameter of a non-linear activation function of the neural network.
14. A computer program product that facilitates creating a deep tensor neural network, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions are executable by a processor to cause the processor to:
create the deep tensor neural network comprising a graph of nodes connected via weighted edges;
in response to the neural network receiving input data and parameters, format the input data and the parameters to generate tensor-formatted input data and tensor-formatted parameters, and wherein at least a portion of the input data is multi-dimensional data;
extract a first portion of features from the tensor-formatted input data in parallel with extraction of a second portion of the features from the tensor-formatted input data; and
training the deep tensor neural network using a Hamiltonian forward propagation process to evolve tensor-structured input data applied to the deep tensor neural network based on a defined tensor-tensor layer evolution rule, wherein the deep tensor neural network generates output data based on the evolving of the tensor-structured input data, wherein the Hamiltonian forward propagation process comprises a multiplication of a circulant function of a first antisymmetric and an unfold function of a second matrix.
15. The computer program product of claim 14 , wherein the program instructions are executable by the processor to cause the processor to:
activate the deep tensor neural network based on a set of non-linear activation functions, wherein the weighted edges and the set of non-linear activation functions operate, based on tensor-tensor functions, to perform the evolution of the tensor-structured input data.
16. The computer program product of claim 14 , wherein the program instructions are executable by the processor to cause the processor to:
apply weights to edges, based on a tensor-tensor product or a multiply type of the tensor-tensor product, to generate the weighted edges.
17. The computer program product of claim 14 , wherein the program instructions are executable by the processor to cause the processor to:
receive a portion of the parameters via an interface from a user, and wherein the portion of the parameters is selected from a group of defined parameters consisting of a defined constant value, a defined network structure, and a non-linear activation function.
18. The computer program product of claim 14 , wherein a portion of the output data is output training data derived from inputting tensor-structured training data to the neural network, and wherein the program instructions are executable by the processor to cause the processor to:
compare the output training data to simulated output data, based on a defined loss function, to determine an update to perform on the neural network to improve operation of the neural network.
19. The computer program product of claim 18 , wherein the program instructions are executable by the processor to cause the processor to:
update the neural network, based on the update, by application of a set of tensor-tensor operations via back-propagation.
20. The computer program product of claim 19 , wherein the update updates a weight of a weighted edge of the weighted edges and updates a parameter of a non-linear activation function of the neural network.