Continuously improving API service endpoint selections via adaptive reinforcement learning
A method includes: receiving, by a processor set, a request for a web-based service; generating, by the processor set, a feature vector including values based on parameters of the request; generating, by the processor set, an endpoint selection vector including plural probabilities corresponding to plural endpoints, wherein the endpoint selection vector is generated using the feature vector with a machine learning model; selecting, by the processor set, one of the plural endpoints based on the plural probabilities; and invoking, by the processor set, the selected endpoint.
1 . A method comprising:
receiving, by a processor set, a request for a web-based service;
generating, by the processor set, a feature vector including values based on parameters of the request;
generating, by the processor set, an endpoint selection vector including plural probabilities corresponding to plural endpoints, wherein the endpoint selection vector is generated using the feature vector with a machine learning model;
selecting, by the processor set, one of the plural endpoints based on a random number and the plural probabilities;
invoking, by the processor set, the selected endpoint;
in response to the selecting one of the plural endpoints, providing data defining the selected one of the plural endpoints to an endpoint selection logger; and
in response to the invoking the selected endpoint, providing quality-of-experience data to the endpoint selection logger, wherein the quality-of-experience data comprises data that quantifies a quality-of-experience associated with the selected endpoint handling the request.
2 . The method of claim 1 , wherein the selected endpoint comprises one of an application program interface (API) endpoint and a web service endpoint.
3 . The method of claim 1 , wherein the machine learning model comprises a deep learning model.
4 . The method of claim 3 , wherein the deep learning model comprises a neural network.
5 . The method of claim 1 , wherein the generating the endpoint selection vector comprises:
inputting the feature vector to the machine learning model; and
receiving the endpoint selection vector as an output of the machine learning model.
6 . The method of claim 1 , wherein the selecting one of the plural endpoints comprises:
generating the random number;
comparing the random number to the plural probabilities; and
in response to the comparing, selecting the one of the plural endpoints based on the random number matching a probability range associated with the one of the plural endpoints.
7 . The method of claim 1 , further comprising training the machine learning model.
8 . The method of claim 7 , wherein the training the machine learning model comprises:
generating a stream of normalized feature vectors, wherein each respective one of the normalized feature vectors is associated with a respective one of plural logged requests;
generating a stream of probabilistic endpoint selection vectors;
determining loss values comprising a respective loss value for each one of the probabilistic endpoint selection vectors;
training the machine learning model using the loss values.
9 . The method of claim 8 , wherein the respective loss value for one of the probabilistic endpoint selection vectors comprises a difference between an expected reward generated by the machine learning model and an expected reward derived from log data.
10 . The method of claim 9 , wherein the expected reward derived from log data comprises a value that represents a quality-of-experience for a pair comprising a particular request feature vector and a particular endpoint.
11 . A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
receive a request for a web-based service;
generate a feature vector including values based on parameters of the request;
generate an endpoint selection vector including plural probabilities corresponding to plural endpoints, wherein the generating the endpoint selection vector comprises providing the feature vector as an input to a deep learning model;
select one of the plural endpoints based on a random number and the plural probabilities; and
invoke the selected endpoint,
wherein training the deep learning model comprises:
generating a stream of normalized feature vectors, wherein each respective one of the normalized feature vectors is associated with a respective one of plural logged requests;
generating a stream of probabilistic endpoint selection vectors;
determining loss values comprising a respective loss value for each one of the probabilistic endpoint selection vectors;
training the deep learning model using the loss values; and
wherein the respective loss value for one of the probabilistic endpoint selection vectors comprises a difference between an expected reward generated by the machine learning model and an expected reward derived from log data.
12 . The computer program product of claim 11 , wherein the selecting one of the plural endpoints comprises:
generating the random number;
comparing the random number to the plural probabilities; and
in response to the comparing, selecting the one of the plural endpoints based on the random number matching a probability range associated with the one of the plural endpoints.
13 . The computer program product of claim 11 , wherein the expected reward derived from log data comprises a value that represents a quality-of-experience for a pair comprising a particular request feature vector and a particular endpoint.
14 . The computer program product of claim 11 , wherein the program instructions are executable to:
in response to the selecting one of the plural endpoints, provide data defining the selected one of the plural endpoints to an endpoint selection logger; and
in response to the invoking the selected endpoint, provide quality-of-experience data to the endpoint selection logger, wherein the quality-of-experience data comprises data that quantifies a quality-of-experience associated with the selected endpoint handling the request.
15 . A system comprising:
a processor set, one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
receive a request for a web-based service;
generate a feature vector including values based on parameters of the request;
generate an endpoint selection vector including plural probabilities corresponding to plural endpoints, wherein the generating the endpoint selection vector comprises providing the feature vector as an input to a deep learning model;
select one of the plural endpoints based on a random number and the plural probabilities; and
invoke the selected endpoint,
wherein the selecting one of the plural endpoints comprises:
generating the random number;
comparing the random number to the plural probabilities; and
in response to the comparing, selecting the one of the plural endpoints based on the random number matching a probability range associated with the one of the plural endpoints.
16 . The system of claim 15 , wherein training the deep learning model comprises:
generating a stream of normalized feature vectors, wherein each respective one of the normalized feature vectors is associated with a respective one of plural logged requests;
generating a stream of probabilistic endpoint selection vectors;
determining loss values comprising a respective loss value for each one of the probabilistic endpoint selection vectors, wherein the respective loss values are based on expected rewards;
training the deep learning model using the loss values.