Neural network error contour generation circuit
A neural network learning mechanism has a device which perturbs analog neurons to measure an error which results from perturbations at different points within the neural network and modifies weights and biases to converge to a target.
1. A neural network error contour generation mechanism comprising a device which perturbs analog neurons to measure an error which results from perturbations at different points within the neural network, wherein the device comprises:
a neuron summer circuit to integrate a perturbation formed of weighted and biased inputs of an error function;
an analog circuit coupled to the neuron summer sampling and holding an activation result of the perturbation to calculate σ′(z) defined as a difference between the activation result and the perturbation; and
a multiplier circuit multiplying σ′(z) by a curl of a cost function to generate output layer errors.
2. The neural network error contour generation mechanism of claim 1 , wherein the neurons are comprised of one or more of: switched charge multipliers, division and current mode summations circuits, and decision circuits.
3. The neural network error contour generation mechanism of claim 1 , comprising means for generating errors for layers below an output layer by using one of switched charge multipliers or division and current mode summations circuits to generate error values by backpropagation through the layers.
4. The neural network error contour generation mechanism of claim 1 , wherein the error caused by a perturbation at each weighted input to a neuron is measured at a respective output of the neural network.
5. The neural network error contour generation mechanism of claim 1 , comprising a neural network update circuit modifying analog weight values to direct the neural network error contour generation mechanism towards a target in response to an error contour generated.
6. The neural network error contour generation mechanism of claim 4 , comprising a neural network update circuit modifying analog weight values to direct the neural network error contour generation mechanism towards a target in response to an error contour generated.
7. The neural network error contour generation mechanism of claim 1 , wherein an error contour generated is stored in an analog memory.
8. The neural network error contour generation mechanism of claim 6 , wherein the error contour generated is stored in an analog memory.
9. The neural network error contour generation mechanism of claim 1 , comprising a circuit modifying analog bias values to direct the neural network error contour generation mechanism towards a target.
10. The neural network error contour generation mechanism of claim 1 , wherein the error is a quadratic difference.