Synthetic scaling applied to shared neural networks
A system processing a stream of input data ordered row by row from a data array has a first integrated circuit (IC) adapted to apply an aperture function to the stream of input data, to produce an output data stream, and a second IC coupled to the first IC, the second IC adapted to manage context from row to row, retaining partial values as computed by the aperture function for each column along a row, and providing the partial values back to the aperture function for subsequent rows as needed to complete output values.
1. A pipelined system processing a stream of input data ordered column by column and row by row from a data array, comprising:
a first integrated circuit (IC) comprising hardware circuitry adapted to simultaneously multiply each datum as received by every weight value of a specific aperture function and to apply a subfunction of the specific aperture function to each datum of the stream of input data as received, producing a partial value of the aperture function for each datum received, and to produce an output value at an output port of the first IC for each column and row position of the data array; and
a second IC coupled to the first IC, the second IC adapted to receive and manage the partial values for each column along each row, passing the partial values through a connected series of first-in-first-out and/or shift registers, providing the partial values back to the first IC for subsequent rows as needed to develop and complete output values;
wherein the output values are a summation of the partial values produced for each input datum for a patch position of the specific aperture function at each application position of the patch in order, and the first IC produces an output value at the output port from a summation of retained values from the second IC and a partial value produced each time a final datum for a patch position is processed, the system producing a stream of output values synchronously with the stream of input values.
2. The system of claim 1 further comprising a first stream of input data from a first data array and a second stream of input data from a second data array, and an interleaving IC combining the first and second data streams into a single interleaved data stream supplied to the first IC.
3. The system of claim 1 further comprising a sampler IC circuit receiving the stream of input data, producing a data stream scaled by a fixed ratio, the full-scale stream and the scaled stream combined by an interleaving IC to the first IC, which produces an interleaved multi-scale output stream.
4. The system of claim 2 wherein the single interleaved data stream is from a first node of a convoluted neural network (CNN) and the interleaved output stream is processed in a second CNN node w/o further downscaling, by the first IC.