Integration of external applications into deep neural networks
During end-to-end training of a Deep Neural Network (DNN), a differentiable estimator subnetwork is operated to estimate a functionality of an external software application. Then, during inference by the trained DNN, the differentiable estimator subnetwork is replaced with the functionality of the external software application, by enabling API communication between the DNN and the external software application.
1. A method comprising:
during end-to-end training of a Deep Neural Network (DNN), operating a differentiable estimator subnetwork to estimate a functionality of an external software application, wherein the estimation comprises learning an Application Programming Interface (API) of the external software application by learning a format and characteristics of input and output parameters of said API, and wherein said end-to-end training trains said DNN to comply with said estimation by said differentiable estimator subnetwork; and
during inference, operating the trained DNN to interact directly, without using said differentiable estimator subnetwork, with said API, to cause said external software application to perform a specified task.
2. The method according to claim 1 , wherein the end-to-end training is with stochastic gradient descent (SGD).
3. The method according to claim 1 , wherein the functionality of the external software application is non-differentiable.
4. The method according to claim 1 , wherein the functionality of the external software application is differentiable.
5. The method according to claim 1 , further comprising operating at least one selector subnetwork to:
determine an API call to the external software application; and
extract suitable API arguments from an input to the DNN.
6. The method according to claim 5 , further comprising operating an adaptation function to:
adapt an output format of the selector subnetwork to an input format required by the API; and
adapt an output format of the API to an input format required by network layers of the DNN during inference, wherein the network layers are higher, in the DNN, than the differentiable estimator subnetwork.
7. The method according to claim 1 , wherein the differentiable estimator subnetwork embeds different data representations into a same vector space, to enable the DNN to handle the different data representations interchangeably.
8. The method according to claim 7 , wherein the different data representations comprise numbers and texts.
9. The method according to claim 1 , wherein:
the differentiable estimator subnetwork is trained prior to being operated during end-to-end training of the DNN; and
the training of the differentiable estimator subnetwork is based on training data that are a generated input for the external software application.
10. The method according to claim 9 , wherein parameters of the differentiable estimator subnetwork are updated during the end-to-end training of the DNN.
11. The method according to claim 1 , wherein:
a loss value is calculated based on a label that is generated by the external software application during the end-to-end training of the DNN; and
the calculated loss value is used in the end-to-end training of the DNN.
12. The method according to claim 1 , wherein:
the differentiable estimator subnetwork is trained prior to being operated during end-to-end training of the DNN;
the training of the differentiable estimator subnetwork is based on training data that are a generated input for the external software application;
a loss value is calculated based on a label that is generated by the external software application during the end-to-end training of the DNN;
the calculated loss value is used in the end-to-end training of the DNN; and
parameters of the differentiable estimator subnetwork are updated during the end-to-end training of the DNN.
13. A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by at least one hardware processor to:
during end-to-end training of a Deep Neural Network (DNN), operate a differentiable estimator subnetwork to estimate a functionality of an external software application, wherein the estimation comprises learning an Application Programming Interface (API) of the external software application by learning a format and characteristics of input and output parameters of said API, and wherein said end-to-end training trains said DNN to comply with said estimation by said differentiable estimator subnetwork; and
during inference, operating the trained DNN to interact directly, without using said differentiable estimator subnetwork, with said API, to cause said external software application to perform a specified task.
14. The computer program product according to claim 13 , wherein:
the program code is further executable to calculate a loss value based on a label that is generated by the external software application during the end-to-end training of the DNN; and
the calculated loss value is used in the end-to-end training of the DNN.
15. The computer program product according to claim 13 , wherein:
the differentiable estimator subnetwork is trained prior to being operated during end-to-end training of the DNN;
the training of the differentiable estimator subnetwork is based on training data that are a generated input for the external software application;
a loss value is calculated based on a label that is generated by the external software application during the end-to-end training of the DNN;
the calculated loss value is used in the end-to-end training of the DNN; and
parameters of the differentiable estimator subnetwork are updated during the end-to-end training of the DNN.
16. A system comprising:
at least one hardware processor; and
a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by said at least one hardware processor to:
during end-to-end training of a Deep Neural Network (DNN), operate a differentiable estimator subnetwork to estimate a functionality of an external software application, wherein the estimation comprises learning an Application Programming Interface (API) of the external software application by learning a format and characteristics of input and output parameters of said API, and wherein said end-to-end training trains said DNN to comply with said estimation by said differentiable estimator subnetwork, and
during inference, operating the trained DNN to interact directly, without using said differentiable estimator subnetwork, with said API, to cause said external software application to perform a specified task.
17. The system according to claim 16 , wherein:
the program code is further executable by said at least one hardware processor to calculate a loss value based on a label that is generated by the external software application during the end-to-end training of the DNN; and
the calculated loss value is used in the end-to-end training of the DNN.