Efficient identification of critical faults in neuromorphic hardware of a neural network
The disclosure provides misclassification-driven training (MDT) that efficiently identifies critical faults in neuromorphic hardware, such as a memristor crossbar. MDT advantageously identifies whether a hardware fault is a critical fault and can be used to limit fault recovery when a hardware fault is not a critical fault. By applying fault-tolerant techniques directed to critical faults, such as only for critical faults, processing overhead of a neural network can be reduced. In one aspect, the disclosure provides a method of identifying critical faults in neuromorphic hardware of a neural network. In one example the method of identifying includes: (1) determining a significant parameter of a trained neural network that impacts classification of a sample of a dataset, (2) obtaining a location of the significant parameter in the neuromorphic hardware, and (3) identifying the location as a critical fault of the neuromorphic hardware.
1 . A method of identifying critical faults in neuromorphic hardware of a neural network, comprising:
determining a significant parameter of a trained neural network that impacts classification of a sample of a dataset by introducing deviated weights to parameters of the trained neural network;
obtaining a location of the significant parameter in the neuromorphic hardware, wherein the neuromorphic hardware is a crossbar of memristors; and
identifying the location as a critical fault of the neuromorphic hardware, wherein the location is a memristor location of the crossbar.
2 . The method as recited in claim 1 , wherein the determining, the obtaining, and the identifying the location are performed iteratively on other samples of the dataset.
3 . The method as recited in claim 2 , wherein the method is performed iteratively until all data points of the dataset are impacted.
4 . The method as recited in claim 1 , wherein the determining further includes deriving a gradient for each of the parameters using backward propagation, determining a significance value of each of the parameters based on the gradient, and identifying a parameter of the parameters having a greatest significance value as the significant parameter.
5 . The method as recited in claim 1 , further comprising creating a deviated parameter value at the location by combining a gradient for the significant parameter with the value of the significant parameter and inferring a type of fault at the location.
6 . The method as recited in claim 5 , wherein the combining is a gradient-ascent step wherein a change in direction of deviation is same as the gradient.
7 . The method as recited in claim 5 , wherein the combining is a gradient-descent when the classification is directed to a targeted class.
8 . The method as recited in claim 5 , wherein the type of fault is a stuck-on fault when the deviated parameter value is within a nominal value range of elements of the neuromorphic hardware.
9 . The method as recited in claim 5 , wherein the type of fault is a stuck-open fault when the deviated parameter value is outside of a nominal value range of elements of the neuromorphic hardware.
10 . A computer program product comprising a non-transitory computer-readable medium having a series of operating instructions stored thereon that directs a processor when executed thereby to perform operations to execute at least some of the steps of the method of claim 1 .
11 . A computing system including a processor that performs at least some of the steps of the method of claim 1 .
12 . A method of training a machine learning (ML) model for classifying critical faults, comprising:
identifying critical faults from a dataset using misclassification-driven training (MDT);
identifying benign faults from the dataset using random fault injections and forward inferencing;
creating a training dataset that includes the critical faults and the benign faults; and
training the ML model using the training dataset.
13 . The method of training as recited in claim 12 , wherein input features for the critical faults include a fault location, a fault type, a parameter significance value, and a deviated parameter value.
14 . The method of training as recited in claim 12 , wherein significance of the benign faults are determined by backpropagating a randomly selected data point from an input dataset.
15 . The method of training as recited in claim 12 , wherein the training of the ML model includes multiple training runs using the training dataset.
16 . The method of training as recited in claim 12 , wherein the training of the ML model includes verifying the benign faults are not identified as a critical fault by injecting the critical faults into the ML model and evaluating whether they are critical.
17 . The method of training as recited in claim 12 , wherein the training of the ML model includes verifying the critical faults cause a misclassification.
18 . A method of identifying faults in a neural network (NN), comprising:
classifying a dataset using a NN;
determining if detection of faults is needed, wherein the faults are hardware faults of the NN and the NN is a crossbar of memristors fabricated on a chip;
performing detection of the faults when needed; and
performing fault recovery of the hardware faults when a detected fault is a critical fault.
19 . The method of identifying faults as recited in claim 18 , wherein the determining is periodically based on a number of rounds of the classifying.
20 . The method of identifying faults as recited in claim 18 , wherein the determining is based on a trigger.
21 . The method of identifying faults as recited in claim 18 , wherein the fault recovery includes remapping.
22 . The method of identifying faults as recited in claim 18 , wherein the fault recovery includes retraining.
23 . A method of manufacturing a chip including a neural network (NN) having a memristor crossbar, comprising:
identifying one or more memristor cells of a crossbar as critical;
allocating fault-tolerant hardware for the one or more memristor cells that are critical; and
fabricating the chip with the fault-tolerant hardware.
24 . The method of manufacturing as recited in claim 23 , wherein the fault-tolerant hardware is at least one spare memristor column corresponding to the one or more memristor cells.
25 . The method of manufacturing as recited in claim 23 , wherein the identifying is based on a classification of critical faults for the neural network and a target application.
26 . The method of manufacturing as recited in claim 23 , wherein the identifying is based on a classification of critical faults for at least one of another neural network and another target application.
27 . A neural network including a chip manufactured according to the method of claim 23 .
28 . An autonomous driving system including a chip manufactured according to the method of claim 23 .
29 . A vision system including a chip manufactured according to the method of claim 23 .