Experiment point recommendation device, experiment point recommendation method, and semiconductor device manufacturing device
For a machine learning model that receives control parameters of a semiconductor processing device and outputs shape parameters that express a processed shape of a semiconductor sample processed by the semiconductor processing device, an experiment point obtaining learning data is recommended. A contribution of each control parameter to the prediction of the machine learning model is evaluated from feature quantity data that is a value of a control parameter of the learning data used for learning of the machine learning model, and the experiment point is recommended based on a stability evaluation and an uncertainty evaluation of the prediction by the machine learning model in a space defined by the control parameters selected based on the contribution as axes.
1. An experiment point recommendation device that recommends an experiment point that is a combination of values of control parameters set in a semiconductor processing device for an experiment to obtain learning data of a machine learning model that receives control parameters of the semiconductor processing device and outputs shape parameters that express a processed shape of a semiconductor sample processed by the semiconductor processing device, the experiment point recommendation device comprising:
a storage device that stores a contribution calculation program, a stability calculation program, an uncertainty calculation program, and an experiment point recommendation program; and
a processor that executes the programs read from the storage device,
wherein the processor executes the contribution calculation program to evaluate the contribution of each control parameter to the prediction of the machine learning model from feature quantity data that is the value of the control parameter of the learning data used for learning of the machine learning model,
the processor executes the stability calculation program to evaluate the stability of prediction by the machine learning model in a first space defined by control parameters selected based on the contribution as axes based on whether or not a change of the value of the selected control parameters causes an abnormal change in the prediction of the machine learning model,
the processor executes the uncertainty calculation program to evaluate the uncertainty of prediction by the machine learning model in a second space defined by the selected control parameters as axes based on a distribution of the feature quantity data in the second space, and
the processor executes the experiment point recommendation program to recommend an experiment point based on the contribution evaluation, the stability evaluation, and the uncertainty evaluation of the selected control parameters to the prediction of the machine learning model.
2. The experiment point recommendation device according to claim 1 ,
wherein in the second space, a contribution conversion value of the control parameter is used as a unit of the value of the selected control parameter as the axis, and
the contribution conversion value is calculated as a value obtained by assigning the value of the shape parameter predicted by the machine learning model based on the contribution evaluation of the control parameter.
3. The experiment point recommendation device according to claim 2 ,
wherein in the first space, the unit of the control parameter is used as the unit of the value of the selected control parameter as the axis, and
the processor executes the experiment point recommendation program to convert the stability evaluation of the prediction by the machine learning model in the first space into the stability evaluation of the prediction by the machine learning model in the second space, and displays an important area based on the contribution evaluation, the stable area based on the stability evaluation, and the uncertain area based on the uncertainty evaluation in the second space.
4. The experiment point recommendation device according to claim 3 ,
wherein the uncertain area is an area included in a range of the uncertainty evaluation specified by the user.
5. The experiment point recommendation device according to claim 3 ,
wherein the second space in which the important area, the stable area, and the uncertain area are displayed is displayed on a user terminal, and
the processor executes the experiment point recommendation program to identify a point specified in the second space displayed on the user terminal by the user as an experiment point.
6. The experiment point recommendation device according to claim 2 ,
wherein in the first space, the unit of the control parameter is used for the unit of the value of the selected control parameter as the axis, and
the processor executes the experiment point recommendation program to convert the stability evaluation of the prediction by the machine learning model in the first space into the stability evaluation of the prediction by the machine learning model in the second space, obtain an integrated score based on the value of the stability evaluation and the value of uncertainty in the second space, and specify an experiment point based on the integrated score.
7. An experiment point recommendation method that recommends an experiment point that is a combination of values of control parameters set in a semiconductor processing device for an experiment to obtain learning data of a machine learning model that receives control parameters of the semiconductor processing device and outputs shape parameters that express a processed shape of a semiconductor sample processed by the semiconductor processing device, the method comprising:
a first step of evaluating the contribution of each control parameter to the prediction of the machine learning model from feature quantity data that is the value of the control parameter of the learning data used for learning of the machine learning model;
a second step of evaluating the stability of prediction by the machine learning model in a first space defined by control parameters selected based on the contribution as axes based on whether or not a change of the value of the selected control parameters causes an abnormal change in the prediction of the machine learning model;
a third step of evaluating the uncertainty of prediction by the machine learning model in a second space defined by the selected control parameters as axes based on a distribution of the feature quantity data in the second space; and
a fourth step of recommending an experiment point based on the contribution evaluation, the stability evaluation, and the uncertainty evaluation of the selected control parameters to the prediction of the machine learning model.
8. The experiment point recommendation method according to claim 7 ,
wherein in the second space, a contribution conversion value of the control parameter is used as a unit of the value of the selected control parameter as the axis, and
the contribution conversion value is calculated as a value obtained by assigning the value of the shape parameter predicted by the machine learning model based on the contribution evaluation of the control parameter.
9. The experiment point recommendation method according to claim 8 ,
wherein in the first space, the unit of the control parameter is used as the unit of the value of the selected control parameter as the axis, and
in the fourth step, the stability evaluation of the prediction by the machine learning model in the first space is converted into the stability evaluation of the prediction by the machine learning model in the second space, and an important area based on the contribution evaluation, a stable area based on the stability evaluation, and an uncertain area based on the uncertainty evaluation are displayed in the second space.
10. The experiment point recommendation method according to claim 9 ,
wherein the uncertain area is an area included in a range of the uncertainty evaluation specified by the user.
11. The experiment point recommendation method according to claim 9 ,
wherein the second space in which the important area, the stable area, and the uncertain area are displayed is displayed on a user terminal, and
in the fourth step, a point specified in the second space displayed on the user terminal by the user is identified as an experiment point.
12. The experiment point recommendation method according to claim 8 ,
wherein in the first space, the unit of the control parameter is used for the unit of the value of the selected control parameter as the axis, and
in the fourth step, the stability evaluation of the prediction by the machine learning model in the first space is converted into the stability evaluation of the prediction by the machine learning model in the second space, an integrated score based on the value of the stability evaluation and the value of uncertainty in the second space is obtained, and an experiment point is specified based on the integrated score.
13. A semiconductor device manufacturing system including a semiconductor processing device and a platform that executes an experiment point recommendation process, which is connected to the semiconductor processing device through a network,
wherein the experiment point recommendation process that recommends an experiment point that is a combination of values of control parameters set in the semiconductor processing device for an experiment to obtain learning data of a machine learning model that receives control parameters of the semiconductor processing device and outputs shape parameters that express a processed shape of a semiconductor sample processed by the semiconductor processing device includes:
a step of evaluating the contribution of each control parameter to the prediction of the machine learning model from feature quantity data that is the value of the control parameter of the learning data used for learning of the machine learning model;
a step of evaluating the stability of prediction by the machine learning model in a first space defined by control parameters selected based on the contribution as axes based on whether or not a change of the value of the selected control parameters causes an abnormal change in the prediction of the machine learning model;
a step of evaluating the uncertainty of prediction by the machine learning model in the second space defined by the selected control parameters as axes based on a distribution of the feature quantity data in the second space; and
a step of recommending an experiment point based on the contribution evaluation, the stability evaluation, and the uncertainty evaluation of the selected control parameters to the prediction of the machine learning model.
14. The semiconductor device manufacturing system according to claim 13 ,
wherein the experiment point recommendation process is executed as an application provided in the platform.