Software defined redundant allocation safety mechanism in an artificial neural network processor
Novel and useful system and methods of several functional safety mechanisms for use in an artificial neural network (ANN) processor. The mechanisms can be deployed individually or in combination to provide a desired level of safety in neural networks. Multiple strategies are applied involving redundancy by design, redundancy through spatial mapping as well as self-tuning procedures that modify static (weights) and monitor dynamic (activations) behavior. The various mechanisms of the present invention address ANN system level safety in situ, as a system level strategy that is tightly coupled with the processor architecture. The NN processor incorporates several functional safety concepts which reduce its risk of failure that occurs during operation from going unnoticed. The mechanisms function to detect and promptly flag and report the occurrence of an error with some mechanisms capable of correction as well. The safety mechanisms cover data stream fault detection, software defined redundant allocation, cluster interlayer safety, cluster intralayer safety, layer control unit (LCU) instruction addressing, weights storage safety, and neural network intermediate results storage safety.
1. A method of end to end failure detection for use in a neural network (NN) processor configured to implement a target neural network, the method comprising:
providing a plurality of unallocated redundant hardware resources in said neural network processor, said unallocated redundant hardware resources including compute, memory, routing, and control elements;
allocating during an offline compilation process via a compiler a main computational path from said plurality of unallocated redundant hardware resources, said main computational path to be protected from end to end failures;
allocating during said offline compilation process via said compiler one or more redundant computational paths from said plurality of unallocated redundant hardware resources different from said plurality of unallocated redundant hardware resources allocated to said main computational path, where said compiler ensures that said main computational path and said one or more redundant computational paths go through and use different resources with no overlap of compute, memory, routing, and control hardware resource elements, wherein said main computational path and said one or more redundant computational paths each function to perform the same calculations for the same target neural network, said one or more redundant computational paths operative to protect said main computational path from end to end failures whereby whole or partial layers of a whole or partial network are allocated;
providing the same input data to said main computational path and said one or more redundant computational paths at the same time;
calculating cyclic redundancy code (CRC) checksums on tensor stream data output from said main computational path and said one or more redundant computational paths;
comparing said CRC checksums from said main computational path and said one or more redundant computational paths with each other;
detecting an error if said calculated CRC checksums do not match; and
wherein said main computational path and said one or more redundant computational paths use different data resources selected from a group consisting of stream managers (SMs), portions of L4 memory allocated to said stream managers, input buffers (IBs), portions of L3 memory, input aligners (IAs), subclusters (SCs), activation processing units (APUs), and output buffers (OBs).
2. The method according to claim 1 , wherein said allocation is determined a priori by a compiler in accordance with desired performance goals for a target neural network.
3. The method according to claim 1 , wherein said main computational path and said one or more redundant computational paths are functionally identical to each other.
4. The method according to claim 1 , wherein said main computational path and said one or more redundant computational paths use different control resources selected from a group consisting of layer controller units (LCUs) and memory management units (MMUs).
5. The method according to claim 1 , wherein hardware resources for said main computational path and said one or more redundant computational paths are allocated in different clusters for each path.
6. The method according to claim 1 , wherein path separation begins within said NN processor at one or more input buffers (IBs) or L3 memory.
7. The method according to claim 1 , further comprising providing built in self-test (BIST) ability where one of said calculated CRC checksums is intentionally altered so as to force generation of an error signal.
8. The method according to claim 1 , wherein hardware resources for said main computational path and said one or more redundant computational paths are allocated on a per layer basis.
9. An apparatus for end to end failure detection for use in a neural network (NN) processor configured to implement a target neural network, comprising:
a plurality of unallocated redundant hardware resources within said neural network processor, said redundant hardware resources including compute, memory, routing, and control elements;
a main computational path allocated during an offline compilation process via a compiler from said plurality of unallocated redundant hardware resources, said main computational path to be protected from end to end failures;
one or more redundant computational paths allocated during said offline compilation process via said compiler from said plurality of unallocated redundant hardware resources different from said plurality of unallocated redundant hardware resources allocated to said main computational path, where said compiler ensures that said main computational path and said one or more redundant computational paths go through and use different resources with no overlap of compute, memory, routing, and control hardware resource elements, wherein said main computational path and said one or more redundant computational paths each function to perform the same calculations for the same target neural network, said one or more redundant computational paths operative to protect said main computational path from end to end failures whereby the same input data is provided to said main computational path and said one or more redundant computational paths at the same time, and whereby whole or partial layers of a whole or partial network are allocated;
one or more cyclic redundancy code (CRC) engines operative to generate CRC checksums on tensor stream data output from said main computational path and said one or more redundant computational paths, compare said CRC checksums from said main computational path and said one or more redundant computational paths with each other, and to detect an error if said calculated CRC checksums do not match; and
wherein said main computational path and said one or more redundant computational paths use different data resources selected from a group consisting of stream managers (SMs), portions of L4 memory allocated to said stream managers, input buffers (IBs), portions of L3 memory, input aligners (IAs), subclusters (SCs), activation processing units (APUs), and output buffers (OBs).
10. The apparatus according to claim 9 , wherein said allocation is determined a priori by a compiler in accordance with desired performance goals for a neural network.
11. The apparatus according to claim 9 , wherein said main computational path and said one or more redundant computational paths are functionally identical to each other.
12. The apparatus according to claim 9 , wherein said main computational path and said one or more redundant computational paths use different control resources selected from a group consisting of layer controller units (LCUs) and memory management units (MMUs).
13. The apparatus according to claim 9 , wherein hardware resources for said main computational path and said one or more redundant computational paths are allocated in different clusters for each path.
14. The apparatus according to claim 9 , wherein path separation begins within said NN processor at one or more input buffers (IBs) or L3 memory.
15. A method of end to end failure detection for use in a neural network processor configured to implement a target neural network, the method comprising:
providing a plurality of unallocated redundant hardware resources in said neural network processor, said unallocated redundant hardware resources including compute, memory, routing, and control elements;
determining a resource allocation scheme in accordance with desired performance goals for a target neural network;
allocating during an offline compilation process via a compiler a plurality of redundant computational tensor data flow paths from said plurality of unallocated redundant hardware resources, where said compiler ensures that said plurality of redundant computational tensor data flow paths go through and use different resources, said plurality of redundant computational tensor data flow paths functionally identical to each other and operative to provide protection from end to end failures by way of said redundancy wherein each redundant computation tensor data flow path uses different unallocated redundant hardware resources with no overlap of compute, memory, routing, and control hardware resource elements, wherein said plurality of redundant computational tensor data flow paths each function to perform the same calculations for the same target neural network, whereby whole or partial layers of a whole or partial network are allocated;
providing the same input data to each of said redundant computational tensor data flow paths at the same time;
comparing cyclic redundancy code (CRC) checksums generated for each redundant computation tensor data flow path and detecting an error if a mismatch is detected; and
wherein said plurality of redundant computational tensor data flow paths use different data resources selected from a group consisting of stream managers (SMs), portions of L4 memory allocated to said stream managers, input buffers (IBs), portions of L3 memory, input aligners (IAs), subclusters (SCs), activation processing units (APUs), and output buffers (OBs).
16. The method according to claim 15 , wherein said plurality of redundant computational tensor data flow paths use different control resources selected from a group consisting of layer controller units (LCUs) and memory management units (MMUs).
17. The method according to claim 15 , wherein hardware resources for said plurality of redundant computational tensor data flow paths are allocated in different clusters for each path.