Modelling and prediction system with auto machine learning in the production of memory devices
To provide more test data during the manufacture of non-volatile memories and other integrated circuits, machine learning is used to generate virtual test values. Virtual test results are interpolated for one set of tests for devices on which the test is not performed based on correlations with other sets of tests. In one example, machine learning determines a correlation study between bad block values determined at die sort and photo-limited yield (PLY) values determined inline during processing. The correlation can be applied to interpolate virtual inline PLY data for all of the memory dies, allowing for more rapid feedback on the processing parameters for manufacturing the memory dies and making the manufacturing process more efficient and accurate. In another set of embodiments, the machine learning is used to extrapolate limited metrology (e.g., critical dimension) test data to all of the memory die through interpolated virtual metrology data values.
1 . A method, comprising:
receiving inline quality control data of test samples from manufacture of an integrated circuit;
receiving post-manufacturing test data of the test samples;
creating a first virtual inline quality control data model for the manufacture of the integrated circuit from the inline quality control data and the post-manufacturing test data;
interpolating virtual inline quality control data for the manufacture of the integrated circuit from the first virtual inline quality control data model and the post-manufacturing test data;
receiving an inline data report for the manufacture of the integrated circuit;
creating a second virtual inline quality control data model for the manufacture of the integrated circuit from the interpolated virtual inline quality control data and the inline data report, where one or both of creating the first virtual inline quality control data model and creating the second virtual inline quality control data model includes:
selecting a machine learning model from a plurality of machine learning models including one or more versions of one or more of the machine learning models with different hyperparameter values; and
interpolating virtual inline quality control data for the manufacture of the integrated circuit from the second virtual inline quality control data model and the inline data report.
2 . The method of claim 1 , wherein selecting a machine learning model from a plurality of machine learning models including one or more versions of one or more of the machine learning models with different hyperparameter values comprises:
applying each of the machine learning models to a data set using a plurality different hyperparameter values for each of the machine learning models; and
selecting the machine learning model and the hyperparameter values for the selected machine learning model based on an amount of error when applied to the data set.
3 . The method of claim 2 , wherein selecting the machine learning model and the hyperparameter values for the selected machine learning model comprises:
ranking combinations of the plurality of machine learning models and the hyperparameters values for the selected machine learning model in order of minimum error.
4 . The method of claim 2 , wherein selecting the machine learning model and the hyperparameter values for the selected machine learning model comprises:
performing a grid search across multiple values for a plurality of hyperparameter parameters for the selected machine learning model.
5 . The method of claim 2 , further comprising:
selecting a first of the plurality of machine learning models; and
determining the data set using the first machine leaning model to rank importance of a plurality of variable.
6 . The method of claim 5 , wherein the first machine learning model is a gradient boosting machine model.
7 . The method of claim 1 , wherein the plurality of machine learning models includes a random forest model.
8 . The method of claim 1 , wherein the plurality of machine learning models includes a generalized linear model.
9 . The method of claim 1 , wherein the plurality of machine learning models includes a gradient boosting machine model.
10 . The method of claim 1 ,
where both of creating the first virtual inline quality control data model and creating the second virtual inline quality control data model include independently selecting a machine learning model from a plurality of machine learning models including one or more versions of one or more of the machine learning models with different hyperparameter values.
11 . The method of claim 1 , wherein receiving the inline quality control data of test samples includes performing tests on the test samples.
12 . The method of claim 11 , wherein receiving the inline quality control data of test samples includes fabricating the test samples.
13 . The method of claim 12 , wherein the test samples of the integrated circuit are fabricated using a first set of processing parameters and the method further comprises:
based on the interpolated virtual inline quality control data for the manufacture of the integrated circuit from the second virtual inline quality control data model and the inline data report, adjusting the first set of processing parameters; and
fabricating the integrated circuit using the adjusted first set of processing parameters.
14 . The method of claim 1 , wherein the post-manufacturing test data is from tests performed as part of a die sort test process.
15 . The method of claim 1 , wherein the post-manufacturing test data is from tests for determining an early failure rate.
16 . The method of claim 1 , wherein the inline quality control data includes transistor gate thickness values.
17 . The method of claim 1 , wherein the inline quality control data includes critical dimension data values.
18 . The method of claim 17 , wherein the integrated circuit is a NAND memory circuit with a three dimensional architecture and the critical dimension data is memory hole critical dimension data.
19 . A method, comprising:
fabricating a plurality of an integrated circuit using a first set of processing parameters;
creating a first virtual inline quality control data model for the fabrication of the integrated circuits from inline quality control data of test samples of the integrated circuit and post-fabrication test data of the test samples;
interpolating virtual inline quality control data for the fabrication of the integrated circuits using the first set of processing parameters from the first virtual inline quality control data model and the post-fabrication test data;
creating a second virtual inline quality control data model for the fabrication of the integrated circuit using the first set of processing parameters from the interpolated virtual inline quality control data and an inline data report for the fabrication of the integrated circuits using the first set of processing parameters, where one or both of creating the first virtual inline quality control data model and creating the second virtual inline quality control data model includes selecting a machine learning model from a plurality of machine learning models including one or more versions of one or more of the machine learning models with different hyperparameter values;
interpolating virtual inline quality control data for the fabrication of the integrated circuit using the first set of processing parameters from the second virtual inline quality control data model and the inline data report;
based on the interpolated virtual inline quality control data for the fabrication of the integrated circuit using the first set of processing parameters from the second virtual inline quality control data model and the inline data report, adjusting the first set of processing parameters; and
fabricating the integrated circuit using the adjusted first set of processing parameters.
20 . A system, comprising:
one or more processors, the one or more processors configured to:
receive, from a fabrication facility, inline quality control data of test samples from manufacture of an integrated circuit;
receive post-manufacturing test data of the test samples;
create a first virtual inline quality control data model for the manufacture of the integrated circuit from the inline quality control data and the post-manufacturing test data;
interpolate virtual inline quality control data for the manufacture of the integrated circuit from the first virtual inline quality control data model and the post-manufacturing test data;
receive, from the fabrication facility, an inline data report for the manufacture of the integrated circuit;
create a second virtual inline quality control data model for the manufacture of the integrated circuit from the interpolated virtual inline quality control data and the inline data report, where one or both of creating the first virtual inline quality control data model and creating the second virtual inline quality control data model includes selecting a machine learning model from a plurality of machine learning models including one or more versions of one or more of the machine learning models with different hyperparameter values; and
provide the second virtual inline quality control data model to the fabrication facility.