Cluster interlayer safety mechanism in an artificial neural network processor
Novel and useful system and methods of functional safety mechanisms for use in an artificial neural network (ANN) processor. The mechanisms can be deployed individually or in combination to provide a desired level of safety in neural networks. Multiple strategies are applied involving redundancy by design, redundancy through spatial mapping as well as self-tuning procedures that modify static (weights) and monitor dynamic (activations) behavior. The NN processor incorporates functional safety concepts which reduce its risk of failure that occurs during operation from going unnoticed. The mechanisms function to detect and promptly flag and report the occurrence of an error with some mechanisms capable of correction as well. The safety mechanisms cover data stream fault detection, software defined redundant allocation, cluster interlayer safety, cluster intralayer safety, layer control unit (LCU) instruction addressing, weights storage safety, and neural network intermediate results storage safety.
1. A method of cluster interlayer neural network tensor data flow path failure detection for use in a neural network processor, the method comprising:
providing a stream of tensors to be written to a memory;
calculating a first cyclic redundancy code (CRC) checksum on each said data tensor over all real features;
writing said tensors together with said first CRC checksums to said memory;
wherein each first CRC checksum is generated across a plurality of real features of a corresponding tensor and stored in said memory as an additional CRC feature of the tensor;
subsequently reading said stream of tensors and corresponding CRC feature of first CRC checksums from said memory and generating a second CRC checksum on said read tensors; and
verifying whether said second CRC checksums match said first CRC checksums and generating an error if a mismatch is detected.
2. The method according to claim 1 , wherein said stream of tensors is provided by at least one of an input buffer (IB) circuit and an activation processing unit (APU) circuit.
3. The method according to claim 1 , further comprising outputting tensors read from said memory to at least one of an input aligner (IA) circuit and an output buffer (OB) circuit.
4. The method according to claim 1 , wherein calculating said first CRC checksum is performed by a CRC generator circuit located in at least one of an input buffer (IB) circuit and an activation processing unit (APU) circuit.
5. The method according to claim 1 , wherein calculating said second CRC checksum is performed by a CRC check circuit located in at least one of an input aligner (IA) circuit and an output buffer (OB) circuit.
6. The method according to claim 1 , wherein said additional feature of said first CRC checksum is as wide as a row width of other features of said tensor.
7. The method according to claim 1 , wherein said first CRC checksum and said second CRC checksum are operative to protect a tensor data flow path selected from a group consisting of input buffer (IB) to input aligner (IA) in a first layer in a cluster, activation processing unit (APU) in a previous layer to IA in a subsequent layer, and APU in the previous layer to output buffer (OB) in a last layer in the cluster.
8. The method according to claim 1 , wherein a granularity of the CRC checksum calculation on said stream of tensors in a previous layer is adapted to match the manner in which a subsequent layer consumes said stream of tensors.
9. A method of cluster interlayer neural network tensor data flow path failure detection for use in a neural network processor, the method comprising:
providing a stream of tensors output from a previous layer to be forwarded to a subsequent layer;
calculating a cyclic redundancy code (CRC) checksum over said tensors wherein each CRC checksum is generated across a plurality of real features of a corresponding tensor;
writing said tensors and together with corresponding CRC checksums to a memory wherein said corresponding CRC checksums are stored as an additional CRC feature of each tensor;
reading said tensors and corresponding CRC feature including CRC checksums from said memory; and
verifying said CRC checksums and raising an error flag if a CRC checksum error is detected.
10. The method according to claim 9 , wherein said stream of tensors is output from an activation processing unit (APU) circuit in a previous layer within a same cluster to an input aligner (IA) circuit via said memory.
11. The method according to claim 9 , wherein said stream of tensors is output from an activation processing unit (APU) circuit in a current layer within a same cluster to an output buffer (OB) circuit via said memory.
12. The method according to claim 9 , wherein said stream of tensors is output from an input buffer in a current cluster to an input aligner (IA) circuit via said memory.
13. The method according to claim 9 , wherein said additional feature of said CRC checksum is as wide as a row width of other features of said tensor.
14. The method according to claim 9 , wherein a granularity of the CRC checksum calculation on said stream of tensors in a previous layer is adapted to match the manner in which a subsequent layer consumes said stream of tensors.
15. An apparatus for cluster interlayer neural network tensor data flow path failure detection for use in a neural network processor, comprising:
a first circuit element operative to provide a stream of tensors from a previous layer to be input to a subsequent layer;
said first circuit element including a first CRC engine operative to calculate a cyclic redundancy code (CRC) checksum over said tensors wherein each CRC checksum is generated across a plurality of real features of a corresponding tensor;
a memory coupled to said first circuit element and operative to store said tensors together with corresponding CRC checksums to said memory wherein said corresponding CRC checksums are stored as an additional CRC feature of each tensor;
a second circuit element coupled to said memory and operative to read said tensors and corresponding CRC features including CRC checksums from said memory; and
said second circuit element including a second CRC engine operative to verify said CRC checksums and raise an error flag if a CRC checksum error is detected.
16. The apparatus according to claim 15 , wherein said first circuit element comprises an activation processing unit (APU) circuit in a previous layer within a same cluster, and said second circuit element comprises an input aligner (IA) circuit.
17. The apparatus according to claim 15 , wherein said first circuit element comprises an activation processing unit (APU) circuit in a current layer within a same cluster, and said second circuit element comprises an output buffer (OB) circuit.
18. The apparatus according to claim 15 , wherein said first circuit element comprises an input buffer in a current cluster, and said second circuit element comprises an input aligner (IA) circuit.
19. The apparatus according to claim 15 , wherein said additional feature of said CRC checksum is as wide as a row width of other features of said tensor.
20. The apparatus according to claim 15 , wherein a granularity of the CRC checksum calculation on said stream of tensors in a previous layer is adapted to match the manner in which a subsequent layer consumes said stream of tensors.