Computing system and method
A technology that can enhance the computing performance of a computing system using reservoir computing (RC), includes a computing system which performs computation using a recurrent neural network (RNN) including an input unit, a reservoir unit, and an output unit. The reservoir unit includes a plurality of nodes circularly connected to each other. The circular connection has a weight matrix for determining a weight between the nodes of the plurality of nodes, in which a weight between the nodes closely arranged on the circle is larger than a weight between the nodes arranged away from each other on the circle. The plurality of nodes each have a g value that is a parameter for designating nonlinearity of an activation function of each of the nodes, and that is set so as to periodically change in a direction on the circle.
1. A computing system configured to perform computation using a recurrent neural network including an input unit, a reservoir unit, and an output unit,
the reservoir unit including a plurality of nodes circularly connected to each other,
the circular connection having a weight matrix for determining a weight between nodes of the plurality of nodes and having, in the weight matrix, a weight between nodes closely arranged on the circle, the weight being larger than a weight between nodes arranged away from each other on the circle,
the plurality of nodes each having a g value that is a parameter for designating nonlinearity of an activation function of each of the nodes, and that is set so as to periodically change in a direction on the circle.
2. The computing system according to claim 1 , wherein
the plurality of nodes have logarithms of the g value, the logarithms being set at even intervals in the direction on the circle.
3. The computing system according to claim 1 , wherein
in the weight matrix, with respect to the nodes, an expected value of an absolute value of a weight between adjacent nodes is larger than an expected value of an absolute value of a weight between nodes adjacent to each other with one node therebetween in the direction on the circle.
4. The computing system according to claim 1 , wherein
the weight in the weight matrix is determined using a random function.
5. The computing system according to claim 1 , wherein
the weight in the weight matrix is determined by computation with a random function array and a Gaussian function array.
6. The computing system according to claim 1 , wherein
the weight matrix is created such that, in the circular connection created with the nodes connected to each other in order of consecutive numbers given to the plurality of nodes, nodes having closer numbers of the numbers have a larger weight.
7. The computing system according to claim 1 , wherein
the reservoir unit and the output unit have a weight designated by an output weight matrix that is determined by least squares regression in which a sum of squares of an error between a linear sum of output values of the plurality of nodes in learning data input and an ideal output value is minimum.
8. The computing system according to claim 1 , wherein
the reservoir unit and the output unit have a weight designated by an output weight matrix that is determined by a Lasso regression method in which a regularization term is added to an error between a linear sum of output values of the plurality of nodes in learning data input and an ideal output value.
9. The computing system according to claim 1 , further comprising:
a computer including
a processing apparatus configured to perform the computation,
a primary storage apparatus and a secondary storage apparatus each configured to store data related to the computation,
an input apparatus configured to input data to the input unit, and
an output apparatus configured to output data from the output unit.
10. The computing system according to claim 1 , wherein
the recurrent neural network is implemented as delay-based reservoir computing.
11. The computing system according to claim 10 , further comprising:
a reservoir device including an optical circuit, wherein
the reservoir device performs part of the computation using the optical circuit.
12. The computing system according to claim 11 , wherein
the optical circuit performs a mask process on an input signal to the reservoir device, and
the mask process includes processing that uses a mask pattern whose strength periodically changes depending on how the g value is set on the circle.
13. A computing method for a computing system configured to perform computation using a recurrent neural network including an input unit, a reservoir unit, and an output unit, the computing method comprising:
creating the reservoir unit, wherein
the reservoir unit includes a plurality of nodes circularly connected to each other,
the circular connection has a weight matrix for determining a weight between nodes of the plurality of nodes and has, in the weight matrix, a weight between nodes closely arranged on the circle, the weight being larger than a weight between nodes arranged away from each other on the circle, and
the plurality of nodes each have a g value that is a parameter for designating nonlinearity of an activation function of each of the nodes, and that is set so as to periodically change in a direction on the circle.