Runtime data collection and monitoring in machine learning systems
A test and measurement system includes a test and measurement instrument configured to receive waveform data from a device under test (DUT) on a manufacturing line, a machine learning system connected to the test and measurement instrument, and one or more processors configured to execute code that causes the one or more processors to: collect optimal tuning parameter data sets from the DUT after the DUT is tuned on the manufacturing line, determine one or more parameter data sets from the optimal tuning parameter data, load the one or more parameter data sets into the DUT, collect waveform data from the DUT for the one or more parameter data sets as training data sets, train the machine learning system using the training data sets, and use the machine learning system after training to produce an output related to the DUT.
1 . A test and measurement system, comprising:
a test and measurement instrument configured to receive waveform data from a device under test (DUT) on a manufacturing line;
a machine learning system connected to the test and measurement instrument;
one or more processors configured to execute code that causes the one or more processors to:
collect optimal tuning parameter data sets from the DUT after the DUT is tuned on the manufacturing line;
determine one or more parameter data sets from the optimal tuning parameter data sets;
load the one or more parameter data sets into the DUT;
collect the waveform data from the DUT for the one or more parameter data sets as training data sets;
train the machine learning system using the training data sets; and
use the machine learning system after training to produce an output related to the DUT; and
a switch configured to select between the DUT being tuned by the manufacturing line, the DUT being tuned by the machine learning system, and the DUT being validated.
2 . The test and measurement system as claimed in claim 1 , wherein the code that causes the one or more processors to determine one or more parameter data sets comprises code that causes the one or more processors to determine a mean parameter data set, and to derive at least one additional parameter data set from the mean parameter data set.
3 . The test and measurement system as claimed in claim 1 , wherein the output related to the DUT comprises one of a validation measurement or a predicted optimal tuning parameter set.
4 . The test and measurement system as claimed in claim 1 , wherein the one or more processors are further configured to execute code that causes the one or more processors to monitor data collection of at least one of the optimal tuning parameter data sets and the waveform data from the DUT.
5 . The test and measurement system as claimed in claim 4 , wherein the code that causes the one or more processors to monitor data collection causes the one or more processors to generate histograms of each tuning parameter in the optimal tuning parameter sets.
6 . The test and measurement system as claimed in claim 5 , wherein the code that causes the one or more processors to generate histograms of each tuning parameter comprises code to cause the one or more processors to analyze the histograms to determine if the machine learning system needs more training.
7 . The test and measurement system as claimed in claim 1 , wherein the one or more processors are further configured to monitor data collection by executing code that causes the one or more processors to:
query a location at which the data is stored;
analyze newest data stored in the location; and
continue collection of data when the newest optimal tuning parameter data meets one or more criteria.
8 . The test and measurement system as claimed in claim 7 , wherein the one or more processors are further configured to execute code that causes the one or more processors to:
diagnose a cause of the data not meeting the one or more criteria; and
provide an update to a user to update the collection of the data.
9 . The test and measurement system as claimed in claim 8 , wherein the one or more processors are further configured to notify a user when the machine learning system needs more training.
10 . The test and measurement system as claimed in claim 1 , wherein the code that causes the one or more processors to collect the waveform data from the DUT comprises code to cause the one or more processors to notify a user when enough of the waveform data has been collected to allow the machine learning system to be trained.
11 . The test and measurement system as claimed in claim 1 , wherein the one or more processors reside in the test and measurement instrument, a computing device on the manufacturing line, the machine learning system, or any combination thereof.
12 . A method of collecting and using data for a machine learning system, comprising:
collecting optimal tuning parameter data sets from a device under test (DUT) after the DUT is tuned on a manufacturing line;
determining one or more parameter data sets from the optimal tuning parameter data;
loading the one or more parameter data sets into the DUT;
collecting waveform data from the DUT for the one or more parameter data sets as training data sets;
training the machine learning system using the training data sets;
using the machine learning system after training to produce an output related to the DUT; and
operating a switch to select between the DUT being tuned by the manufacturing line, the DUT being tuned by the machine learning system, and the DUT being validated.
13 . The method as claimed in claim 12 , wherein determining the one or more parameter data sets comprises determining a mean parameter data set and deriving at least one additional parameter data set from the mean parameter data set.
14 . The method as claimed in claim 12 , wherein the output related to the DUT comprises one of a validation measurement or a predicted optimal tuning parameter set.
15 . The method as claimed in claim 12 , further comprising monitoring data collection of at least one of the optimal tuning parameter data sets and the waveform data from the DUT.