Test system for detecting faults in multiple devices of the same type
Various embodiments relate to a method of testing a plurality of devices of the same type wherein each of the plurality of devices of the same type include a built-in self-test device, including: randomly generating, by a processor, stimulus parameters; applying, by the built-in self-test devices, the generated stimulus parameters N times to the plurality of devices of the same type; measuring, by the plurality of devices of the same type, a response of the plurality of devices of the same type to the generated stimulus parameters to produce M×N response outputs, where M is a number of the plurality of devices of the same type; calculating, by the processor, a defect likelihood for a test set of the plurality of identical devices based upon a mean of a reference set of the plurality of identical devices response outputs, a mean of the test set response outputs, a standard deviation of reference set response outputs, and a standard deviation of the test set response outputs; determining, by the processor, that the defect likelihood for the test set is greater than a first threshold value; applying, by the processor, an initial step of a directed random search algorithm to update stimulus parameters in response to determining that the defect likelihood is greater than the first threshold; applying, by the built-in self-test devices, the updated stimulus parameters N times to the plurality of devices of the same type; measuring, by the plurality of devices of the same type, a response of the plurality of devices of the same type to the updated stimulus parameters to produce M×N updated response outputs; calculating, by the processor, a defect likelihood for the test set based upon a mean of the reference set updated response outputs, a mean of the test set updated response outputs, a standard deviation of reference set updated response outputs, and a standard deviation of the test set updated response outputs; and determining, by the processor, that the defect likelihood for the test set is greater than a second threshold, wherein the second threshold is greater than the first threshold.
1 . A method of testing a plurality of devices of a same type wherein each of the plurality of devices of the same type include a built-in self-test (BIST) device, comprising:
randomly generating, by a processor of the BIST device, a stimulus signal based on a stimulus parameter;
applying, by the BIST device, the stimulus signal N times to the plurality of devices of the same type;
measuring, by the processor, a response of the plurality of devices of the same type to the stimulus signal to produce M×N response outputs, wherein M is a number of the plurality of devices of the same type;
calculating, by the processor, a defect likelihood for a test set of the plurality of devices of the same type based upon a mean of a reference set of the plurality of devices of the same type response outputs, a mean of a test set response outputs, a standard deviation of reference set response outputs, and a standard deviation of a test set response outputs;
determining, by the processor, that the defect likelihood for the test set is greater than a first threshold;
applying, by the processor, an initial operation of a directed random search algorithm to obtain an updated stimulus parameter in response to determining that the defect likelihood is greater than the first threshold, wherein the updated stimulus parameter is used to generate updated stimulus signal to be applied to the plurality of devices of the same type to improve detection of a defect;
applying, by the BIST devices, the updated stimulus signal N times to the plurality of devices of the same type;
measuring, by the processor, a response of the plurality of devices of the same type to the updated stimulus signal to produce M×N updated response outputs;
calculating, by the processor, a defect likelihood for the test set based upon a mean of a reference set updated response outputs, a mean of a test set updated response outputs, a standard deviation of a reference set updated response outputs, and a standard deviation of a test set updated response outputs; and
determining, by the processor, that the defect likelihood for the test set is greater than a second threshold, wherein the second threshold is greater than the first threshold.
2 . The method of claim 1 , further comprising indicating a defect in response to determining that the defect likelihood is greater than the second threshold.
3 . The method of claim 1 , further comprising:
in response to determining that the defect likelihood is greater than the second threshold repeating:
applying the updated stimulus signal N times to the plurality of devices of the same type;
measuring the response of the plurality of devices of the same type to the updated stimulus signal to produce M×N second response outputs;
calculating the defect likelihood for a test set based upon a mean of the reference set updated response outputs, a mean of the test set updated response outputs, a standard deviation of reference set updated response outputs, and a standard deviation of the test set updated response outputs;
determining that the defect likelihood for the test set is greater than the second threshold; and
indicating a defect in response to determining that a repeated defect likelihood is greater than the second threshold.
4 . The method of claim 1 , wherein the directed random search algorithm is a Nelder-Mead search algorithm.
5 . The method of claim 1 , wherein calculating the defect likelihood for the test set further includes:
calculating a first moment statistical test r mean as
r
mean
=
mean
(
test
set
)
-
mean
(
ref
set
)
std
(
ref
set
)
,
wherein std is a sample standard deviation; and
calculating a second moment statistical test r var as
r
var
=
std
(
test
set
)
std
(
ref
set
)
wherein an estimated defect likelihood s is determined from a mean value of a standard normal distribution as: s=max(abs(r2s m(r mean ) ), abs(r2s v(r var )) , wherein r2s_m( ) and r2s_v( ) are functions that map the values r mean and r var to values s mean and s var with cdf m(r mean ) =normcdf(s mean ), cdf v(r var ) =normcdf(s var ), and cdf m ( ) and cdf v ( ) are cumulative distribution functions of ratios r mean and r var respectively, and normcdf(s) is a cumulative distribution function of the standard normal distribution.
6 . The method of claim 1 , further comprising:
in response to determining that an updated estimated defect likelihood is less than the first threshold repeating:
randomly generating stimulus signal based on the stimulus parameter;
applying the stimulus signal N times to the plurality of devices of the same type; and
measuring the response of the plurality of devices of the same type to the stimulus signal to produce M×N response outputs; and
in response to determining that the updated estimated defect likelihood is greater than the first threshold:
applying an iteration operation of the directed random search algorithm to update the stimulus parameter.
7 . A test system for testing a plurality of devices of a same type, comprising:
a plurality of built-in self-test (BIST) devices, wherein each of the plurality of devices of the same type include one of the plurality of BIST devices;
at least one processor; and
at least one memory storing instructions, that in response to determining that executed by the at least one processor, cause the test system at least to:
randomly generate a stimulus signal based on a stimulus parameter;
apply, by the one of the plurality of BIST devices, the stimulus signal N times to the plurality of devices of the same type;
measure, by the plurality of devices of the same type, a response of the plurality of devices of the same type to the stimulus signal to produce M×N response outputs, wherein M is a number of the plurality of devices of the same type;
calculating a defect likelihood for a test set of the plurality of devices of the same type based upon a mean of a reference set of the plurality of devices of the same type response outputs, a mean of a test set response outputs, a standard deviation of reference set response outputs, and a standard deviation of a test set response outputs;
determine that the defect likelihood for the test set is greater than a first threshold;
apply an initial operation of a directed random search algorithm to obtain an updated stimulus parameter in response to determining that the defect likelihood is greater than the first threshold, wherein the updated stimulus parameter is used to generate updated stimulus signal applied to the plurality of devices of the same type to improve detection of a defect;
apply, by the BIST devices, the updated stimulus signal N times to the plurality of devices of the same type;
measure, by the plurality of devices of the same type, a response of the plurality of devices of the same type to the updated stimulus signal to produce M×N updated response outputs;
calculating a defect likelihood for the test set based upon a mean of a reference set updated response outputs, a mean of a test set updated response outputs, a standard deviation of a reference set updated response outputs, and a standard deviation of a test set updated response outputs; and
determining that the defect likelihood for the test set is greater than a second threshold, wherein the second threshold is greater than the first threshold.
8 . The test system of claim 7 , wherein the at least one memory storing instructions cause the test system at least to indicate a defect in response to determining that the defect likelihood is greater than the second threshold.
9 . The test system of claim 7 , wherein the at least one memory storing instructions cause the test system at least to:
in response to determining that the defect likelihood is greater than the second threshold repeating:
apply the updated stimulus signal N times to the plurality of devices of the same type;
measure the response of the plurality of devices of the same type to the updated stimulus signal to produce M×N second response outputs;
calculate the defect likelihood for the test set based upon a mean of the reference set updated response outputs, a mean of the test set updated response outputs, a standard deviation of reference set updated response outputs, and a standard deviation of the test set updated response outputs;
compare the defect likelihood for the test set to the second threshold; and
indicate a defect in response to determining that a repeated defect likelihood is greater than the second threshold.
10 . The test system of claim 7 , wherein the directed random search algorithm is a Nelder-Mead search algorithm.
11 . The test system of claim 7 , wherein calculating the defect likelihood for the test set further includes:
calculating a first moment statistical test r mean as
r
mean
=
mean
(
test
set
-
mean
(
ref
set
)
std
(
ref
set
)
,
wherein std is a sample standard deviation; and
calculating a second moment statistical test r var as
r
var
=
std
(
test
set
)
std
(
ref
set
)
wherein an estimated defect likelihood s is determined from a mean value of a standard normal distribution as: s=max(abs(r2s m(r mean ) ), abs(r2s v(r var ) ), wherein r2s_m( ) and r2s_v( ) are functions that map the values r mean and r var to values s mean and s var with cd f m(r mean ) =normcdf(s mean ), cdf v(r var ) =normcdf(s var ), and cdf v ( ) and cdf v ( ) are cumulative distribution functions of ratios r mean and r var respectively, and normcdf(s) is a cumulative distribution function of the standard normal distribution.
12 . The test system of claim 7 , wherein the at least one memory storing instructions cause the test system at least to:
in response to determining that an updated estimated defect likelihood is less than the first threshold repeat:
randomly generate stimulus signal based on the stimulus parameter;
apply the stimulus signal N times to the plurality of devices of the same type; and
measure the response of the plurality of devices of the same type to the stimulus signal to produce M×N response outputs; and
in response to determining that the updated estimated defect likelihood is greater than the first threshold:
apply an iterations operation of the directed random search algorithm to update the stimulus parameter.
13 . A method of testing a plurality of devices of a same type wherein each of the plurality of devices of the same type include a built-in self-test device, comprising:
generating, by a discriminator, a stimulus signal based on a stimulus parameter, wherein the discriminator is a machine learning model;
applying, by the built-in self-test device, the stimulus signal to the plurality of devices of the same type;
measuring a response of the plurality of devices of the same type to the stimulus signal to produce M response outputs, wherein M is a number of the plurality of devices of the same type;
generating, by a generator, M−1 weights, wherein the generator is a machine learning model;
calculating a first response output from a first identical device of the plurality of devices of the same type as a weighted average of the other M−1 response outputs using the M−1 weights associated with M−1 other devices of the same type;
determining a difference between an estimated first response output and the first response output;
updating the discriminator and the generator using the first response output in response to determining that the difference is less than a threshold;
updating the discriminator and the generator using the estimated first response output in response to determining that the difference is greater than the threshold; and
controlling, by a controller the discriminator and the generator to act as a generative adversarial network wherein the discriminator searches for stimulus parameter that indicate defects and wherein the generator to generate M−1 weights that indicate defects.
14 . The method of claim 13 , further comprising indicating a defect in the first identical device associated with the first response output in response to determining that the difference is greater than the threshold.
15 . The method of claim 13 , further comprising outputting, by the discriminator, device control parameters to the plurality of devices of the same type.
16 . A test system for testing a plurality of devices of a same type wherein each of the plurality of devices of the same type include a built-in self-test device, comprising:
a plurality of built-in self-test (BIST) devices, wherein each of the plurality of devices of the same type include one of the plurality of BIST devices; and
at least one processor; and
at least one memory storing instructions, that in response to determining that executed by the at least one processor, cause the test system at least to:
generate stimulus parameters by a discriminator implemented as a machine learning model in the at least one processor;
apply the stimulus parameters to the plurality of BIST devices;
receive M response outputs from the plurality of BIST devices, wherein M is a number of the plurality of devices of the same type;
generate M−1 weights by a generator implemented as a machine learning model in the at least one processor;
estimate a first response output from a first identical device of the plurality of devices of the same type as a weighted average of the other M−1 response outputs using the M−1 weights associated with M−1 other devices of the same type;
determine a difference between an estimated first response output and the first response output;
update the discriminator and the generator using the first response output in response to determining that the difference is less than a threshold;
update the discriminator and the generator using the estimated first response output in response to determining that the difference is greater than the threshold; and
control the discriminator and the generator to act as a generative adversarial network wherein the discriminator searches for stimulus parameters that indicate defects and wherein the generator to generate M−1 weights that indicate defects.
17 . The test system of claim 16 , wherein the at least one memory storing instructions cause the test system at least to indicate a defect in the first identical device associated with the first response output in response to determining that the difference is greater than the threshold.
18 . The test system of claim 16 , wherein the at least one memory storing instructions cause the test system at least to output device control parameters generated by the discriminator to the plurality of devices of the same type.