Selection of content delivery networks using agents
In some embodiments, a method determines an input value for a request for content and retrieves a plurality of states for a plurality of content delivery networks. The state includes a component that is based on variance. A plurality of scores for a plurality of content delivery networks is generated based on the input value and the plurality of states. A score for a content delivery network is generated using a parameter value for the component and the input value. The method selects a content delivery network from the plurality of content delivery networks based on the score for the content delivery network and outputs information for the selected content delivery network to cause the selected content delivery network to service the request to deliver the content.
1. A method comprising:
determining, by a computing device, an input value for a request for content;
retrieving, by the computing device, a plurality of states for a plurality of content delivery networks, wherein a state includes a first parameter and a second parameter based on historical observations of delivery of content for the respective content delivery network;
generating, by the computing device, a plurality of scores for the plurality of content delivery networks based on the input value and the plurality of states, wherein a score for a content delivery network is generated using a first component that represents an exploitation value from the first parameter and the second parameter, a second component that represents a variance from the first parameter and the second parameter, and the input value;
selecting, by the computing device, a content delivery network from the plurality of content delivery networks based on the score for the content delivery network; and
outputting, by the computing device, information for the selected content delivery network to cause the selected content delivery network to service the request to deliver the content.
2. The method of claim 1 , wherein generating the plurality of scores comprises:
generating the score for each content delivery network in the plurality of content delivery networks based on a respective state for the content delivery network and the input value.
3. The method of claim 1 , wherein the first parameter decreases as a number of historical observations for a respective content delivery network increases.
4. The method of claim 1 , wherein the score is based on a prediction of an upper bound.
5. The method of claim 1 , wherein the request for content is received from a client, and content delivery networks in the plurality of content delivery networks are able to deliver the content to the client.
6. The method of claim 1 , wherein the input value comprises a set of input values based on a context associated with the request.
7. The method of claim 6 , wherein the context is associated with a client that sent the request and receives a delivery of the content.
8. The method of claim 1 , wherein generating the plurality of scores comprises:
using a plurality of agents to generate the plurality of scores, wherein each content delivery network is associated with a respective agent.
9. The method of claim 8 , wherein a state is stored for each of the plurality of agents.
10. The method of claim 1 , further comprising:
receiving feedback from a delivery of the content by the selected content delivery network; and
adjusting the first parameter or the second parameter based on the feedback.
11. The method of claim 10 , wherein:
the feedback is based on a performance metric that is measured based on the delivery of the content by the selected content delivery network.
12. The method of claim 11 , wherein the feedback comprises whether a rebuffer occurred during the delivery of the content by the selected content delivery network.
13. The method of claim 12 , wherein the first parameter or the second parameter is changed to a value to predict higher scores when no rebuffer occurs.
14. The method of claim 10 , wherein adjusting the first parameter or the second parameter is performed in real time when the feedback is received.
15. The method of claim 10 , wherein adjusting the first parameter or the second parameter is performed by an operation to adjust the first parameter or the second parameter from a first value to a second value using the feedback.
16. The method of claim 1 , wherein selecting the content delivery network comprises:
selecting a highest ranked score from the plurality of scores.
17. The method of claim 1 , wherein:
the first parameter represents historical selections of content delivery networks in the plurality of content delivery network; and
the second parameter is based on occurrences for a metric.
18. The method of claim 1 , wherein the exploitation value is based on a mean of the first parameter and the second parameter.
19. A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for:
determining an input value for a request for content;
retrieving a plurality of states for a plurality of content delivery networks, wherein a state includes a first parameter and a second parameter based on historical observations of delivery of content for the respective content delivery network;
generating a plurality of scores for the plurality of content delivery networks based on the input value and the plurality of states, wherein a score for a content delivery network is generated using a first component that represents an exploitation value from the first parameter and the second parameter, a second component that represents a variance from the first parameter and the second parameter, and the input value;
selecting a content delivery network from the plurality of content delivery networks based on the score for the content delivery network; and
outputting information for the selected content delivery network to cause the selected content delivery network to service the request to deliver the content.
20. An apparatus comprising:
one or more computer processors; and
a computer-readable storage medium comprising instructions for controlling the one or more computer processors to be operable for:
determining an input value for a request for content;
retrieving a plurality of states for a plurality of content delivery networks, wherein a state includes a first parameter and a second parameter based on historical observations of delivery of content for the respective content delivery network;
generating a plurality of scores for the plurality of content delivery networks based on the input value and the plurality of states, wherein a score for a content delivery network is generated using a first component that represents an exploitation value from the first parameter and the second parameter, a second component that represents a variance from the first parameter and the second parameter, and the input value;
selecting a content delivery network from the plurality of content delivery networks based on the score for the content delivery network; and
outputting information for the selected content delivery network to cause the selected content delivery network to service the request to deliver the content.