Model integration apparatus, model integration method, computer-readable storage medium storing a model integration program, inference system, inspection system, and control system

1 . A model integration apparatus comprising a processor configured with a program to perform operations comprising:

operation as a model collecting unit configured to collect trained learning models from a plurality of learning apparatuses, wherein each of the plurality of learning apparatuses collects local learning data, and constructs a trained learning model that has acquired a predetermined capability by performing machine learning using the collected local learning data, the trained learning models each having a structure comprising a common portion having a common structure and an integration range comprising computational parameters that are shared between the trained learning models, the integration range determined based on information regarding the respective structures of the trained learning models, and at least one of the trained learning models having the structure comprising different structures in a portion other than the common portion;

operation as a range adjusting unit configured to determine for each of the collected trained learning models, a range to which integration processing is to be performed as the integration range in the common portion included in each of the collected trained learning models;

operation as an integration processing unit configured to execute integration processing comprising, for each of the collected trained learning models, integrating results of machine learning reflected in the integration range set in the common portion, with respect to the integration ranges set in all of the collected trained learning models; and

operation as a model updating unit configured to distribute a result of the integration processing to each learning apparatus in the plurality of learning apparatuses, and update each of the trained learning models retained by the learning apparatuses by causing each learning apparatus in the plurality of learning apparatuses to apply the result of the integration processing to the integration range in the respective trained learning model, wherein

the common portion of each trained learning model comprises a plurality of computation modules, and

the processor is configured with the program to perform operations as the range adjusting unit such that determining the integration range in the common portion comprises:

acquiring an output from each computation module included in the common portion by inputting an evaluation sample to each trained learning model;

calculating at least one of: a similarity and a correlation in the acquired output between the corresponding computation modules of the respective trained learning models; and

designating the computation module for which the calculated at least one of the similarity and the correlation satisfies a predetermined condition as the integration range.

2 . The model integration apparatus according to claim 1 , wherein

the structure of each trained learning model is given by a template, and

the processor is configured with the program such that operation as the range adjusting unit comprises determining the integration range based on the template.

3 . The model integration apparatus according to claim 1 , wherein the processor is configured with the program to perform operations such that operation as the range adjusting unit comprises optimizing the integration range such that that predetermined capability of the trained learning models improves after execution of the integration processing relative to that before execution of the integration processing.

4 . The model integration apparatus according to claim 3 , wherein

the processor is configured with the program to perform operations such that:

operation as the range adjusting unit comprises designating a provisional integration range in the common portion; and

operation as the integration processing unit comprises executing integration processing of integrating results of the machine learning reflected in the designated provisional integration range, with respect to the trained learning models, and

the processor is configured with the program to perform operations as the range adjusting unit such that optimizing the integration range comprises:

acquiring a first result obtained by letting the trained learning models before execution of the integration processing exert the predetermined capability on an evaluation sample;

acquiring a second result obtained by letting the trained learning models after execution of the integration processing exert the predetermined capability on the evaluation sample;

determining whether the predetermined capability of the trained learning models has improved after execution of the integration processing relative to before execution of the integration processing, by comparing the first result with the second result; and

determining the provisional integration range as integration range, responsive to determining that the predetermined capability of the trained learning models improves after executing the integration processing.

5 . The model integration apparatus according to claim 2 , wherein

the processor is configured with the program to perform operations such that:

operation as the range adjusting unit comprises designating a plurality of provisional integration ranges in the common portion; and

operation as the integration processing unit comprises executing, for each provisional integration range, integration processing of integrating results of the machine learning reflected in the designated provisional integration range with respect to the trained learning models, and

the processor is configured with the program to perform operations as the range adjusting unit such that optimizing the integration range comprises:

acquiring, for each provisional integration range, a result obtained by letting the trained learning models after execution of the integration processing exert the predetermined capability on an evaluation sample;

specifying a provisional integration range for which the predetermined capability of the trained learning models after execution of the integration processing is highest, based on the acquired results; and

determining the specified provisional integration range as the integration range.

6 . The model integration apparatus according to claim 1 , wherein

the processor is configured with the program to perform operations further comprising operation as a grouping unit configured to assign each of the plurality of learning apparatuses to at least one of a plurality of groups,

the processor is configured with the program to perform operations such that:

operation as the model collecting unit comprises collecting the trained learning models from a plurality of learning apparatuses belonging to the same group;

operation as the range adjusting unit comprises determining the integration range in a common portion included in each of the collected trained learning models in the same group;

operation as the integration processing unit comprises executing integration processing of integrating results of the machine learning reflected in the determined integration range, with respect to the trained learning models, in the same group; and

operation as the model updating unit comprises distributing a result of the integration processing to the learning apparatuses in the same group, and update the trained learning models retained by these learning apparatuses by causing the learning apparatuses to apply the result of the integration processing to the integration range in the trained learning models.

7 . The model integration apparatus according to claim 6 , wherein the processor is configured with the program to perform operations such that operation as the grouping unit comprises distributing a list of the plurality of groups to each learning apparatus, to cause the learning apparatus to select at least one group from the plurality of groups on the list, and to assign the learning apparatuses to their selected at least one group.

8 . The model integration apparatus according to claim 6 , wherein the processor is configured with the program to perform operations such that operation as the grouping unit comprises:

acquiring attribute data regarding the predetermined capability from each learning apparatus;

performing clustering of the attribute data acquired from the learning apparatuses; and

assigning each of the plurality of learning apparatuses to at least one of the plurality of groups based on the clustering result.

9 . The model integration apparatus according to claim 6 , wherein the processor is configured with the program to perform operations such that operation as the grouping unit comprises optimizing the assignment of the learning apparatuses to the plurality of groups such that the predetermined capability of the trained learning models improves after execution of the integration processing relative to that before execution of the integration processing.

10 . The model integration apparatus according to claim 9 , wherein the processor is configured with the program to perform operations such that:

operation as the grouping unit comprises performing a plurality of provisional groupings that differ in assignment of the learning apparatuses to the plurality of groups;

operation as the range adjusting unit comprises determining, for each provisional grouping, the integration range in a common portion included in each collected trained learning model within the same group; and

operation as the integration processing unit comprises executing, for each provisional grouping, integration processing of integrating results of the machine learning reflected in the determined integration range, with respect to the trained learning models in the same group, and

the processor is configured with the program to perform operations as the range adjusting unit such that optimizing the assignment of the learning apparatuses comprises:

acquiring, for each provisional grouping, a result obtained by letting the trained learning models after execution of the integration processing in the same group exert the predetermined capability on an evaluation sample;

specifying a provisional grouping for which the predetermined capability of the trained learning models after execution of the integration processing is highest, in the same group, based on the acquired results; and

assigning each of the plurality of learning apparatuses to at least one of the plurality of groups according to the specified provisional grouping.

11 . The model integration apparatus according to claim 1 , wherein the processor is configured with the program to perform operations as the integration processing unit such that integrating results of the machine learning reflected in the integration range comprises obtaining an average or a sum of values of a computation parameter included in the integration range.

12 . The model integration apparatus according to claim 1 , wherein

the processor is configured with the program to perform operations further comprising operation as a weight setting unit configured to set weights for the trained learning models, wherein the weights determine priorities of the trained learning models in the integration processing, and

the processor is configured with the program to perform operations as the integration processing unit such that integrating results of the machine learning reflected in the integration range comprises obtaining an average or a sum of values of a computation parameter included in the integration range of the respective trained learning models, after subjecting the values to weighting using the set weights.

13 . The model integration apparatus according to claim 12 , wherein the processor is configured with the program to perform operations such that operation as the weight setting unit comprises setting the weights such that the weight increases as the number of pieces of the local learning data used for the machine learning increases.

14 . The model integration apparatus according to claim 12 , wherein

the local learning data comprises specific learning data regarding a specific element in the predetermined capability, and

the processor is configured with the program to perform operations such that operation as the weight setting unit comprises setting the weights such that the weight increases as the ratio of the specific learning data included in the local learning data used for the machine learning increases.

15 . The model integration apparatus according to claim 1 , wherein the processor is configured with the program to perform operations as the integration processing unit such that the integration of results of the machine learning reflected in the integration range is performed by a secret calculation performed by a method comprising one of: a homomorphic encryption method; a secret sharing method; and a method compliant with international standard ISO/IEC 19592-2:2017.

16 . An inference system comprising:

a plurality of learning apparatuses; and

a model integration apparatus, wherein

each learning apparatus comprises a first processor configured with a first program to perform operations comprising:

collecting local learning data; and

constructing a trained learning model that has acquired a predetermined inference capability by machine learning using the collected local learning data,

each of the trained learning models having a structure comprising a common portion having a common structure and an integration range comprising computational parameters that are shared between the trained learning models, the integration range determined based on information regarding the respective structures of the trained learning models, and at least one of the trained learning models having the structure comprising different structures in a portion other than the common portion, and

the model integration apparatus comprises a second processor configured with a second program to perform operations comprising:

collecting the trained learning models from the plurality of learning apparatuses;

determining for each of the collected trained learning models, a range to which integration processing is to be performed as the integration range in the common portion included in each of the collected trained learning models;

executing integration processing comprising, for each of the collected trained learning models, integrating results of the machine learning reflected in the integration range set in the common portion, with respect to the integration ranges set in all of the collected trained learning models; and

distributing a result of the integration processing to each learning apparatus in the plurality of learning apparatuses, and updating each of the trained learning models retained by the learning apparatuses by causing each learning apparatus in the plurality of learning apparatuses to apply the result of the integration processing to the integration range in the respective trained learning models, wherein

the common portion of each trained learning model comprises a plurality of computation modules, and

determining the integration range in the common portion comprises:

acquiring an output from each computation module included in the common portion by inputting an evaluation sample to each trained learning model;

calculating at least one of: a similarity and a correlation in the acquired output between the corresponding computation modules of the respective trained learning models; and

designating the computation module for which the calculated at least one of the similarity and the correlation satisfies a predetermined condition as the integration range.

17 . A model integration method executed by a computer, comprising:

collecting trained learning models from a plurality of learning apparatuses, wherein each of the plurality of learning apparatuses collects local learning data, and constructs a trained learning model that has acquired a predetermined capability by performing machine learning using the collected local learning data, each of the trained learning models having a structure comprising a common portion having a common structure and an integration range comprising computational parameters that are shared between the trained learning models, the integration range determined based on information regarding the respective structures of the trained learning models, and at least one of the trained learning models having the structure comprising different structures in a portion other than the common portion;

operation as a range adjusting unit configured to determine for each of the collected trained learning models, a range to which integration processing is to be performed as the integration range in the common portion included in each of the collected trained learning models;

executing integration processing comprising, for each of the collected trained learning models, integrating results of machine learning reflected in the integration range set in the common portion, with respect to the integration ranges set in all of the collected trained learning models; and

distributing a result of the integration processing to each learning apparatus in the plurality of learning apparatuses, and updating each of the trained learning models retained by the learning apparatuses by causing each learning apparatus in the plurality of learning apparatuses to apply the result of the integration processing to the integration range in the respective trained learning model, wherein

the common portion of each trained learning model comprises a plurality of computation modules, and

determining the integration range in the common portion comprises:

acquiring an output from each computation module included in the common portion by inputting an evaluation sample to each trained learning model;

calculating at least one of: a similarity and a correlation in the acquired output between the corresponding computation modules of the respective trained learning models; and

designating the computation module for which the calculated at least one of the similarity and the correlation satisfies a predetermined condition as the integration range.

18 . A non-transitory computer-readable storage medium storing a model integration program, which when read and executed, causes a computer to perform operations comprising:

collecting trained learning models from a plurality of learning apparatuses, wherein each of the plurality of learning apparatuses collects local learning data, and constructs a trained learning model that has acquired a predetermined capability by performing machine learning using the collected local learning data, each of the trained learning models having a structure comprising a common portion having a common structure and an integration range comprising computational parameters that are shared between the trained learning models, the integration range determined based on information regarding the respective structures of the trained learning models, and at least one of the trained learning models having the structure comprising different structures in a portion other than the common portion;

executing integration processing comprising, for each of the collected trained learning models, integrating results of machine learning reflected in the integration range set in the common portion, with respect to the integration ranges set in all of the collected trained learning models; and

determining for each of the collected trained learning models, a range to which integration processing is to be performed as the integration range in the common portion included in each of the collected trained learning models;

distributing a result of the integration processing to each learning apparatus in the plurality of learning apparatuses, and updating each of the trained learning models retained by the learning apparatuses by causing each learning apparatus in the plurality of learning apparatuses to apply the result of the integration processing to the integration range in the respective trained learning model, wherein

the common portion of each trained learning model comprises a plurality of computation modules, and

determining the integration range in the common portion comprises:

acquiring an output from each computation module included in the common portion by inputting an evaluation sample to each trained learning model;

calculating at least one of: a similarity and a correlation in the acquired output between the corresponding computation modules of the respective trained learning models; and

designating the computation module for which the calculated at least one of the similarity and the correlation satisfies a predetermined condition as the integration range.

19 . An inspection system comprising:

a plurality of inspection apparatuses; and

a model integration apparatus, wherein

each inspection apparatus comprises a first processor configured with a first program to perform operations comprising:

collecting local learning data including image data of an image of a product; and

constructing a trained learning model that has acquired a capability of determining a state of the product in an image of the image data by performing machine learning using the collected local learning data,

each trained learning model having a structure comprising a common portion having a common structure and an integration range comprising computational parameters that are shared between the trained learning models, the integration range determined based on information regarding the respective structures of the trained learning models, and at least one of the trained learning models having the structure comprising different structures in a portion other than the common portion

the model integration apparatus comprises a second processor configured with a second program to perform operations comprising:

collecting the trained learning models from the plurality of inspection apparatuses,

determining for each of the collected trained learning models, a range to which integration processing is to be performed as the integration range in the common portion included in each of the collected trained learning models;

executing integration processing comprising, for each of the collected trained learning models, integrating results of the machine learning reflected in the integration range set in the common portion, with respect to the integration ranges set in all of the collected trained learning models, and

distributing a result of the integration processing to each inspection apparatus in the plurality of inspection apparatuses, and updating each of the trained learning models retained by the inspection apparatuses by causing each inspection apparatus in the plurality of inspection apparatuses to apply the result of the integration processing to the integration range in the respective trained learning models, wherein

the common portion of each trained learning model comprises a plurality of computation modules, and

determining the integration range in the common portion comprises:

acquiring an output from each computation module included in the common portion by inputting an evaluation sample to each trained learning model;

calculating at least one of: a similarity and a correlation in the acquired output between the corresponding computation modules of the respective trained learning models; and

designating the computation module for which the calculated at least one of the similarity and the correlation satisfies a predetermined condition as the integration range.