Cluster intralayer safety mechanism in an artificial neural network processor
Novel and useful system and methods of functional safety mechanisms for use in an artificial neural network (ANN) processor. The mechanisms can be deployed individually or in combination to provide a desired level of safety in neural networks. Multiple strategies are applied involving redundancy by design, redundancy through spatial mapping as well as self-tuning procedures that modify static (weights) and monitor dynamic (activations) behavior. The NN processor incorporates several functional safety concepts which reduce its risk of failure that occurs during operation from going unnoticed. The mechanisms function to detect and promptly flag and report the occurrence of an error with some mechanisms capable of correction as well. The safety mechanisms cover data stream fault detection, software defined redundant allocation, cluster interlayer safety, cluster intralayer safety, layer control unit (LCU) instruction addressing, weights storage safety, and neural network intermediate results storage safety.
1 . A method of cluster intralayer tensor data flow path failure detection for use in a neural network processor, the method comprising:
providing a physical neural network intralayer tensor data flow hardware circuit path for performing regular artificial neural network (ANN) computation processing on a stream of regular input data tensors and weights that are part of the ANN to be implemented;
compiler generation of tensor test data and weights as well as expected test output based thereon a priori via numeric emulation of said physical neural network intralayer tensor data flow hardware circuit path and used solely for safety and protection of the neural network processor;
temporal multiplexing of said physical neural network intralayer tensor data flow hardware circuit path for both regular inference of the regular input data tensors as well as fault detection utilizing said tensor test data and weights;
injecting said tensor test data and weights into said physical neural network intralayer tensor data flow hardware circuit path during a fault detection window after regular ANN inference processing of the regular input data tensors has paused thereby providing end-to-end intralayer fault coverage without interfering with normal ANN processing throughput;
calculating a test output based on said test sensors and test weights during the fault detection window; and
comparing said test output with the expected test output and generating an error if a mismatch is detected.
2 . The method according to claim 1 , further comprising:
calculating a test cyclic redundancy code (CRC) checksum on said test output; and
comparing said test CRC checksum with a precalculated test CRC checksum and generating an error if a mismatch is detected.
3 . The method according to claim 1 , wherein said physical neural network intralayer tensor data flow hardware circuit path consisting of an input aligner (IA), subcluster (SC), and activation processing unit (APU) circuits.
4 . The method according to claim 1 , wherein said tensor test data is injected into said physical neural network intralayer tensor data flow hardware circuit path periodically or a periodically.
5 . The method according to claim 1 , wherein said tensor test data comprises a pseudorandom binary sequence generated dynamically.
6 . The method according to claim 1 , wherein said tensor test data is determined a priori and stored in a memory.
7 . An apparatus for cluster intralayer tensor data flow path failure detection for use in a neural network processor, comprising:
a physical neural network intralayer tensor data flow hardware circuit path for performing regular artificial neural network (ANN) computation processing on a stream of regular input data tensors and weights that are part of the ANN to be implemented;
a first hardware circuit to receive and store tensor test data and weights as well as expected test output based thereon a priori generated by a compiler via numeric emulation of said physical neural network intralayer tensor data flow hardware circuit path and used solely for safety and protection of the neural network processor;
a second hardware circuit operative to temporally multiplex said physical neural network intralayer tensor data flow hardware circuit path for both regular inference of the regular input data tensors as well as fault detection utilizing said tensor test data and weights;
a third hardware circuit operative to inject said tensor test data and weights into said physical neural network intralayer tensor data flow hardware circuit path during a fault detection window after regular ANN inference processing of the regular input data tensors has paused thereby providing end-to-end intralayer fault coverage without interfering with normal ANN processing throughput;
a fourth hardware circuit operative to calculate a test output based on said test sensors and test weights during the fault detection window; and
a fifth hardware circuit operative to compare said test output with the expected test output and generating an error if a mismatch is detected.
8 . The apparatus according to claim 7 , further comprising:
a sixth hardware circuit incorporating a cyclic redundancy code (CRC) engine and operative to calculate a test CRC checksum on said test output; and
a comparator circuit operative to compare said test CRC checksum with a test CRC checksum calculated a priori and to generate an error if a mismatch is detected.
9 . The apparatus according to claim 7 , wherein said physical neural network intralayer tensor data flow hardware circuit path comprises a tensor data flow path consisting of an input aligner (IA), subcluster (SC), and activation processing unit (APU) circuits.
10 . The apparatus according to claim 7 , wherein said third hardware circuit is operative to inject said tensor test data into said physical neural network intralayer tensor data flow hardware circuit path periodically or a periodically.
11 . The apparatus according to claim 7 , wherein said tensor test data are generated and configured by a software compiler.
12 . The apparatus according to claim 7 , wherein said tensor test data comprises a pseudorandom binary sequence generated dynamically.
13 . The apparatus according to claim 7 , wherein said tensor test data are determined a priori and stored in a memory.