Multi-layer neural network
A multi-layer artificial neural network having at least one high-speed communication interface and N computational layers is provided. N is an integer larger than 1. The N computational layers are serially connected via the at least one high-speed communication interface. Each of the N computational layers respectively includes a computation circuit and a local memory. The local memory is configured to store input data and learnable parameters for the computation circuit. The computation circuit in the ith computational layer provides its computation results, via the at least one high-speed communication interface, to the local memory in the (i+1)th computational layer as the input data for the computation circuit in the (i+1)th computational layer, wherein i is an integer index ranging from 1 to (N−1).
1. A multi-layer neural network, comprising:
at least one high-speed communication interface;
N computational layers serially connected via the at least one high-speed communication interface, N being an integer larger than 1, each of the N computational layers respectively comprising a computation circuit and a local memory, the local memory being configured to store input data and learnable parameters for the computation circuit, the computation circuit in the ith computational layer providing its computation results, via the at least one high-speed communication interface, to the local memory in the (i+1)th computational layer as the input data for the computation circuit in the (i+1)th computational layer, wherein i is an integer index ranging from 1 to (N−1), wherein the computation circuit in a jth computational layer is configured to perform computation based on both real data and dummy data, the dummy data filled into an input matrix of the jth computational layer to expand a size of the input matrix of the jth computational layer to be equal to a size of an input matrix of the (j−1)th computational layer; and
a controller coupled to the computation circuit and configured to invalidate computation results generated based on the dummy data and validate computation results generated completely based on real data.
2. The multi-layer neural network of claim 1 , wherein the N computational layers are implemented by at least N identical unit chips.
3. The multi-layer neural network of claim 2 , wherein the N computational layers comprise a convolutional layer, this convolutional layer comprises M identical unit chips connected in parallel, and M is a positive integer larger than 1.
4. The multi-layer neural network of claim 2 , wherein the N computational layers comprise a fully-connected layer, this fully-connected layer comprises P identical unit circuits connected in parallel and working as P neurons, and P is a positive integer larger than 1.
5. The multi-layer neural network of claim 1 , wherein the computation circuit in the ith computational layer sequentially generates Q computation results, Q is an integer larger than 1; when generating the qth computation result among the Q computation results, the computation circuit provides the (q−1)th computation result to the local memory in the (i+1)th computational layer at the same time, wherein q is an integer index ranging from 2 to Q.
6. The multi-layer neural network of claim 1 , wherein the N computational layers operate simultaneously based on a pipeline architecture.
7. The multi-layer neural network of claim 1 , wherein the high-speed communication interface comprises a serializer-deserializer (SerDes) or a radio frequency interface (RFI).
8. The multi-layer neural network of claim 1 , wherein the ith computational layer and the (i+1)th computational layer are configured to be located in the same integrated-circuit chip, in two different integrated-circuit chips but the same package, or in two different packages.