Recurrent neural circuits

1. A method comprising:

providing a computer system, the computer system including at least a processor and a memory, the memory comprising at least an operating system;

executing a process in the memory, the process comprising:

providing a recurrent circuit model; and converting the recurrent circuit model into a recurrent neural network that can be fit with gradient descent, wherein converting comprises applying a first stage and applying a second stage,

wherein applying the first stage comprises:

suppressing feedforward encodings Z by non-negative interactions between units in H[t−1], a feedback gated recurrent unit (fGRU) hidden state from a previous timestep;

computing suppressive interactions with a kernel WS∈R E×E×K×K wherein E describes a spatial extent of horizontal connections on a single timestep, S is another population of neurons without hidden states, R is a set of real numbers and K is a number of channels of the Kernel;

convolving the kernel a gated version of a persistent hidden state H[t- 1 ]; and

computing a gate activity G f by applying a sigmoid nonlinearity to a convolution of the kernel U F ∈R 1×1×K×K with H[t−1], transforming its activity into the range [0, 1].

2. The method of claim 1 wherein additive and multiplicative forms of suppression are controlled by parameters μ, α∈R K , respectively, wherein μ and α control linear and non-linear forms of suppressive computations.

3. The method of claim 2 wherein applying the second stage comprises:

applying additive and multiplicative facilitation to an instantaneous activity S, the kernels W F ∈R E×E×K×K controlling facilitation interactions;

scaling additive and multiplicative forms of facilitation are by the parameters v, ω∈R K , respectively , wherein v and ω control linear and non-linear forms of facilitative computations; and

computing a gate activity G F to update the persistent recurrent activity H.

4. The method of claim 3 wherein computing the gate activity G F comprises applying a sigmoid to a convolution of the kernel U F ∈R 1×1×K×K with S, wherein the gate activity G F updates H[t] by interpolating H[t−1] with the candidate activity ft the H[t] taken as a fGRU output activity.

5. The method of claim 4 wherein the fGRU output hidden state is either passed to the next convolutional layer or used to compute top-down connections.