Electronic control unit testing optimization
A computer-implemented method for implementing electronic control unit (ECU) testing optimization includes capturing, within a neural network model, input-output relationships of a plurality of ECUs operatively coupled to a controller area network (CAN) bus within a CAN bus framework, including generating the neural network model by pruning a fully-connected neural network model based on comparisons of maximum values of neuron weights to a threshold, reducing signal connections of a plurality of collected input signals and a plurality of collected output signals based on connection weight importance, ranking importance of the plurality of collected input signals based on the neural network model, generating, based on the ranking, a test case execution sequence for testing a system including the plurality of ECUs to identify flaws in the system, and initiating the test case execution sequence for testing the system.
1. A computer-implemented method for implementing electronic control unit (ECU) testing optimization, comprising:
capturing, within a neural network model, input-output relationships of a plurality of ECUs operatively coupled to a controller area network (CAN) bus within a CAN bus framework, including generating the neural network model by pruning a fully-connected neural network model based on comparisons of maximum values of neuron weights to a threshold;
reducing signal connections of a plurality of collected input signals and a plurality of collected output signals based on connection weight importance;
ranking importance of the plurality of collected input signals based on the neural network model; and
generating, based on the ranking, a test case execution sequence for testing a system including the plurality of ECUs to identify flaws in the system; and
initiating the test case execution sequence for testing the system, wherein the test case execution sequence executes a first test case corresponding to a first collected input signal earlier than a second test case corresponding to a second collected input signal having a lower ranking than the first collected input signal.
2. The method as recited in claim 1 , wherein the plurality of ECUs is associated with an autonomous vehicle.
3. The method as recited in claim 1 , further comprising training the neural network model until a predetermined accuracy rate is met and for each correctly classified example at least one condition is satisfied, the at least one condition including a max absolute sample error < , wherein is a user defined parameter corresponding to a target classification rate.
4. The method as recited in claim 1 , wherein the fully-connected neural network model corresponds to a deep feed-forward neural network model including an input layer, a first hidden layer, a second hidden layer and an output layer, and wherein pruning the fully-collected neural network model further includes:
for each w l m , ν p m and d h m , determining whether
max
h
max
p
v
p
m
d
h
m
<
ϵ
and removing ν p m and d h m in response to determining that
max
h
max
p
v
p
m
d
h
m
<
ϵ
;
for each ν p m , determining whether max|ν p m |<∈ and removing ν p m in response to determining that max|ν p m |<∈;
for each d h m , determining whether max|d h m |<∈; and removing d h m in response to determining that max|d h m |<∈; and
removing w l m with a smallest|w l m ν p m d h m | in response to failing to find any weights satisfying
max
h
max
p
v
p
m
d
h
m
<
ϵ
,
max|ν p m |<∈, and |d h m |<∈;
wherein w l m represents a neuron weight from the input layer to the first hidden layer, d h m represents a neuron weight from the first hidden layer to the second hidden layer, ν p m represents a neuron weight from the second hidden layer to the output layer, subscripts l, h and p represent weight indices, superscript m marks the variables as the neuron weights in the neural network model, and E is the threshold having a value between 0 and 1, inclusive.
5. The method as recited in claim 4 , wherein ranking the importance of the plurality of input signals further includes:
computing a product Σ h Σ p |w l m ν p m d h m |;
sorting input nodes in order of the product in ascending order to generate a sorted list; and
assigning ranks according to positions within the sorted list.
6. The method as recited in claim 1 , further comprising producing a segmentation for continuous input signals of the plurality of input signals.
7. The method as recited in claim 6 , wherein producing the segmentation for continuous input signals further includes:
selecting an input signal variable corresponding to an input signal from a neural network model;
selecting an output signal variable corresponding to an output signal from the neural network model;
performing a clustering analysis to cluster the input signal into a plurality of segments; and
representing an activation value of each of the plurality of segments as an average of activation values within the segment.
8. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method for implementing electronic control unit (ECU) testing optimization, the method performed by the computer comprising:
capturing, within a neural network model, input-output relationships of a plurality of ECUs operatively coupled to a controller area network (CAN) bus within a CAN bus framework, including generating the neural network model by pruning a fully-connected neural network model based on comparisons of maximum values of neuron weights to a threshold;
reducing signal connections of a plurality of collected input signals and a plurality of collected output signals based on connection weight importance;
ranking importance of the plurality of collected input signals based on the neural network model;
generating, based on the ranking, a test case execution sequence for testing a system including the plurality of ECUs to identify flaws in the system; and
initiating the test case execution sequence for testing the system, wherein the test case execution sequence executes a first test case corresponding to a first collected input signal earlier than a second test case corresponding to a second collected input signal having a lower ranking than the first collected input signal.
9. The computer program product as recited in claim 8 , wherein the plurality of ECUs is associated with an autonomous vehicle.
10. The computer program product as recited in claim 8 , wherein the method further includes training the neural network model until a predetermined accuracy rate is met and for each correctly classified example at least one condition is satisfied, the at least one condition including a max absolute sample error < , wherein is a user defined parameter corresponding to a target classification rate.
11. The computer program product as recited in claim 8 , wherein pruning the fully-collected neural network model further includes:
for each w l m , ν p m and d h m , determining whether
max
h
max
p
v
p
m
d
h
m
<
ϵ
and removing ν p m and d h m in response to determining that
max
h
max
p
v
p
m
d
h
m
<
ϵ
;
for each ν p m , determining whether max|ν p m |<★ and removing ν p m in response to determining that max|ν p m |<∈;
for each d h m , determining whether max|d h m |<∈; and removing d h m in response to determining that max|d h m |<∈; and
removing w l m with a smallest|w l m ν p m d h m | in response to failing to find any weights satisfying
max
h
max
p
v
p
m
d
h
m
<
ϵ
,
max|ν p m |<∈, and |d h m |<∈;
wherein w l m represents a neuron weight from the input layer to the first hidden layer, d h m represents a neuron weight from the first hidden layer to the second hidden layer, ν p m represents a neuron weight from the second hidden layer to the output layer, subscripts l, h and p represent weight indices, superscript m marks the variables as the neuron weights in the neural network model, and E is the threshold having a value between 0 and 1, inclusive.
12. The computer program product as recited in claim 11 , wherein ranking the importance of the plurality of input signals further includes:
computing a product Σ h Σ p |w l m ν p m d h m |;
sorting input nodes in order of the product in ascending order to generate a sorted list; and
assigning ranks according to positions within the sorted list.
13. The computer program product as recited in claim 8 , wherein the method further includes producing a segmentation for continuous input signals of the plurality of input signals.
14. The computer program product as recited in claim 13 , wherein producing the segmentation for continuous input signals further includes:
selecting an input signal variable corresponding to an input signal from a neural network model;
selecting an output signal variable corresponding to an output signal from the neural network model;
performing a clustering analysis to cluster the input signal into a plurality of segments; and
representing an activation value of each of the plurality of segments as an average of activation values within the segment.
15. A system for implementing electronic control unit (ECU) testing optimization, comprising:
a memory device storing program code; and
at least one processor device operatively coupled to the memory device and configured to execute program code stored on the memory device to:
capture, within a neural network model, input-output relationships of a plurality of ECUs operatively coupled to a controller area network (CAN) bus within a CAN bus framework by generating the neural network model by pruning a fully-connected neural network model based on comparisons of maximum values of neuron weights to a threshold;
reduce signal connections of a plurality of collected input signals and a plurality of collected output signals based on connection weight importance;
rank importance of the plurality of collected input signals based on the neural network model;
generate, based on the ranking, a test case execution sequence for testing a system including the plurality of ECUs to identify flaws in the system; and
initiate the test case execution sequence for testing the system, wherein the test case execution sequence executes a first test case corresponding to a first collected input signal earlier than a second test case corresponding to a second collected input signal having a lower ranking than the first collected input signal.
16. The system as recited in claim 15 , wherein the plurality of ECUs is associated with an autonomous vehicle.
17. The system as recited in claim 15 , wherein the at least one processor device is further configured to execute program code stored on the memory device to train the neural network model until a predetermined accuracy rate is met and for each correctly classified example at least one condition is satisfied, the at least one condition including a max absolute sample error < , wherein is a user defined parameter corresponding to a target classification rate.
18. The system as recited in claim 15 , wherein the at least one processor device is further configured to:
prune the fully-collected neural network model by:
for each w l m , ν p m and d h m , determining whether
max
h
max
p
v
p
m
d
h
m
<
ϵ
and removing ν p m and d h m in response to determining that
max
h
max
p
v
p
m
d
h
m
<
ϵ
;
for each ν p m , determining whether max|ν p m |<★ and removing ν p m in response to determining that max|ν p m |<∈;
for each d h m , determining whether max|d h m |<∈ and removing d h m in response to determining that max|d h m |<∈; and
removing w l m with a smallest|w l m ν p m d h m | in response to failing to find any weights satisfying max max|ν p m d h m |<∈, max|ν p m |<∈, and |d h m |<∈; and rank the importance of the plurality of input signals by:
computing a product Σ h Σ p |w l m ν p m d h m ;
sorting input nodes in order of the product in ascending order to generate a sorted list; and
assigning ranks according to positions within the sorted list;
wherein w l m represents a neuron weight from the input layer to the first hidden layer, d h m represents a neuron weight from the first hidden layer to the second hidden layer, ν p m represents a neuron weight from the second hidden layer to the output layer, subscripts l, h and p represent weight indices, superscript m marks the variables as the neuron weights in the neural network model, and E is the threshold having a value between 0 and 1, inclusive.
19. The system as recited in claim 15 , wherein the at least one processor device is further configured to execute program code stored on the memory device to produce a segmentation for continuous input signals of the plurality of input signals.
20. The system as recited in claim 19 , wherein the at least one processor device is further configured to produce the segmentation for continuous input signals by:
selecting an input signal variable corresponding to an input signal from a neural network model;
selecting an output signal variable corresponding to an output signal from the neural network model;
performing a clustering analysis to cluster the input signal into a plurality of segments; and
representing an activation value of each of the plurality of segments as an average of activation values within the segment.