Neural adapter for classical machine learning (ML) models
Solutions for adapting machine learning (ML) models to neural networks (NNs) include receiving an ML pipeline comprising a plurality of operators; determining operator dependencies within the ML pipeline; determining recognized operators; for each of at least two recognized operators, selecting a corresponding NN module from a translation dictionary; and wiring the selected NN modules in accordance with the operator dependencies to generate a translated NN. Some examples determine a starting operator for translation, which is the earliest recognized operator having parameters. Some examples connect inputs of the translated NN to upstream operators of the ML pipeline that had not been translated. Some examples further tune the translated NN using backpropagation. Some examples determine whether an operator is trainable or non-trainable and flag related parameters accordingly for later training. Some examples determine whether an operator has multiple corresponding NN modules within the translation dictionary and make an optimized selection.
1 . A system for adapting machine learning (ML) models to neural networks (NNs), the system comprising:
a processor; and
a computer-readable medium storing instructions that are operative upon execution by the processor to:
receive an ML pipeline comprising a plurality of operators;
parse the ML pipeline;
determine a plurality of recognized operators within the plurality of operators;
for each of the plurality of recognized operators, determine trainability;
for each trainable operator of the plurality of recognized operators, identify a set of related parameters;
flag the identified set of related parameters for the trainable operators as a set of trainable parameters;
for each non-trainable operator of the plurality of recognized operators, identify a set of related parameters;
flag the identified set of related parameters for the non-trainable operators as a set of non-trainable parameters;
for each of the plurality of recognized operators, select a corresponding NN module from a translation dictionary; and
wire, using a NN wiring component, the selected NN modules to generate a translated NN.
2 . The system of claim 1 , wherein the instructions are further operative to:
extract parameter values from the ML pipeline; and
copy the extracted parameters values into the translated NN.
3 . The system of claim 1 , wherein the instructions are further operative to:
track the set of trainable parameters; and
tune the translated NN using the set of trainable parameters and training data.
4 . The system of claim 1 , wherein the instructions are further operative to:
track the set of non-trainable parameters; and
exclude the set of non-trainable parameters from tuning of the translated NN.
5 . The system of claim 1 , wherein the translated NN is tuned using backpropagation.
6 . The system of claim 1 , wherein one of the plurality of recognized operators has a plurality of corresponding NN modules, and wherein the instructions are further operative to:
generate selection rules for selecting a NN module from the plurality of corresponding NN modules.
7 . The system of claim 1 , wherein the instructions are further operative to:
generate additional translations for the translation dictionary.
8 . A method of adapting machine learning (ML) models to neural networks (NNs), the method comprising:
receiving an ML pipeline comprising a plurality of operators;
parsing the ML pipeline;
determining a plurality of recognized operators within the plurality of operators;
for each of the plurality of recognized operators, determining trainability;
for each trainable operator of the plurality of recognized operators, identifying a set of related parameters;
flagging the identified set of related parameters for the trainable operators as a set of trainable parameters;
for each non-trainable operator of the plurality of recognized operators, identifying a set of related parameters;
flagging the identified set of related parameters for the non-trainable operators as a set of non-trainable parameters;
for each of the plurality of recognized operators, selecting a corresponding NN module from a translation dictionary; and
wiring the selected NN modules to generate a translated NN.
9 . The method of claim 8 , further comprising:
extracting parameter values from the ML pipeline; and
copying the extracted parameters values into the translated NN.
10 . The method of claim 8 , further comprising:
tracking the set of trainable parameters; and
tuning the translated NN using the set of trainable parameters and training data.
11 . The method of claim 8 , further comprising:
tracking the set of non-trainable parameters; and
excluding the set of non-trainable parameters from tuning of the translated NN.
12 . The method of claim 8 , wherein the translated NN is tuned using backpropagation.
13 . The method of claim 8 , wherein one of the plurality of recognized operators has a plurality of corresponding NN modules, and further comprising:
generating selection rules for selecting a NN module from the plurality of corresponding NN modules.
14 . The method of claim 8 , further comprising:
generating additional translations for the translation dictionary.
15 . One or more computer storage devices having computer-executable instructions stored thereon for adapting machine learning (ML) models to neural networks (NNs), which, on execution by a computer, cause the computer to perform operations comprising:
receiving an ML pipeline comprising a plurality of operators;
parsing the ML pipeline;
determining a plurality of recognized operators within the plurality of operators;
for each of the plurality of recognized operators, determining trainability;
for each trainable operator of the plurality of recognized operators, identifying a set of related parameters;
flagging the identified set of related parameters for the trainable operators as a set of trainable parameters;
for each non-trainable operator of the plurality of recognized operators, identifying a set of related parameters;
flagging the identified set of related parameters for the non-trainable operators as a set of non-trainable parameters;
for each of the plurality of recognized operators, selecting a corresponding NN module from a translation dictionary; and
wiring the selected NN modules to generate a translated NN.
16 . The one or more computer storage devices of claim 15 , wherein the operations further comprise:
extracting parameter values from the ML pipeline; and
copying the extracted parameters values into the translated NN.
17 . The one or more computer storage devices of claim 15 , wherein the operations further comprise:
tracking the set of trainable parameters; and
tuning the translated NN using the set of trainable parameters and training data.
18 . The one or more computer storage devices of claim 15 , wherein the operations further comprise:
tracking the set of non-trainable parameters; and
excluding the set of non-trainable parameters from tuning of the translated NN.
19 . The one or more computer storage devices of claim 15 , wherein the translated NN is tuned using backpropagation.
20 . The one or more computer storage devices of claim 15 , wherein one of the plurality of recognized operators has a plurality of corresponding NN modules, and wherein the operations further comprises:
generating selection rules for selecting a NN module from the plurality of corresponding NN modules.