Content selection metric determination for a content distribution system
A content distribution system determines a content selection metric that allows the content distribution system to evaluate whether a content provider maximizes its net content value by stating the true content value of presenting content items by the content distribution system. The content distribution system determines net content values of presented content items and re-performs content selection processes based on modified content value offers. The content distribution system then determines sets of alternative net content values and compares the alternative net content values to the corresponding net content values to determine if the content provider can achieve a greater net content value with a content value offer that does not match the stated content value of presenting the content value. The content distribution system then determines a content selection metric based on the compared net content values and alternative net content values.
1. A method comprising:
receiving, at a content distribution system, a content item and a content value offer from a content provider, the content value offer indicating an amount of content value offered by the content provider for presenting the content item;
determining a stated content value for the content item based on the content value offer;
ranking a set of content items for presentation to a user of the content distribution system based on respective stated content values of the set of content items, the set of content items comprising the received content item from the content provider;
determining a net content value to the content provider of presenting the content item to the user, the net content value being determined based on the stated content value, the net content value associated with the ranking;
determining a set of alternative net content values for the content provider, each alternative net content value is determined by:
calculating a modified content value offer based on a modifier,
re-ranking the set of content items for presentation to the user of the content distribution system based on the modified content value offer, and
determining the alternative net content value based on the stated content value, the alternative net content value associated with the re-ranking; and
determining a content selection metric based on the net content value and the set of alternative net content values, the content selection metric representing a measure of whether the ranking obtains selection results consistent with the stated value of a content item.
2. The method of claim 1 , wherein the stated content value is the content value offer received from the content provider.
3. The method of claim 1 , wherein determining the stated content value comprises: receiving the stated content value from the content provider.
4. The method of claim 1 , wherein performing the content selection process comprises: applying a content selection model.
5. The method of claim 1 , wherein ranking the set of content items for presentation comprises: comparing the content value offer to content value offers from other content providers.
6. The method of claim 1 , wherein the modifier is of a set of modifiers, the set of modifiers comprising one or more values by which the content value offer is modified.
7. The method of claim 1 , wherein the modifier is of a set of modifiers, the set of modifiers comprising one or more percentages by which the content value offer is modified.
8. The method of claim 1 , wherein re-ranking the set of content items for presentation comprises: holding one or more of the respective stated content values of the set of content items constant.
9. The method of claim 1 , wherein determining the content selection metric comprises: determining a count of the set of alternative net content values that exceed the net content value.
10. The method of claim 1 , wherein determining the content selection metric comprises: determining a percentage of the set of alternative net content values that exceed the net content value.
11. The method of claim 1 , wherein the content value offer represents an amount of real-world currency to be paid to the content distribution system.
12. The method of claim 1 , wherein the content selection metric is determined based on a plurality of net content values and a plurality of sets of alternative net content values, each net content value of the plurality of net content corresponding to a set of alternative net content values of the plurality of sets of alternative net content values.
13. A non-transitory, computer-readable medium comprising instructions that, when executed by a processor, cause the processor to:
receive, at a content distribution system, a content item and a content value offer from a content provider, the content value offer indicating an amount of content value offered by the content provider for presenting the content item;
determine a stated content value for the content item based on the content value offer;
rank a set of content items for presentation to a user of the content distribution system based on respective stated content values of the set of content items, the set of content items comprising the received content item from the content provider;
determine a net content value to the content provider of presenting the content item to the user, the net content value being determined based on the stated content value, the net content value associated with the ranking;
determine a set of alternative net content values for the content provider, each alternative net content value is determined by:
calculate a modified content value offer based on a modifier,
re-rank the set of content items for presentation to the user of the content distribution system based on the modified content value offer, and
determine the alternative net content value based on the stated content value, the alternative net content value associated with the re-ranking; and
determine a content selection metric based on the net content value and the set of alternative net content values, the content selection metric representing a measure of whether the ranking obtains selection results consistent with the stated value of a content item.
14. The computer-readable medium of claim 13 , wherein the modifier is of a set of modifiers, the set of modifiers comprising one or more values by which the content value offer is modified.
15. The computer-readable medium of claim 13 , wherein the modifier is of a set of modifiers, the set of modifiers comprising one or more percentages by which the content value offer is modified.
16. The computer-readable medium of claim 13 , wherein re-ranking the set of content items for presentation comprises: holding one or more of the respective stated content values of the set of content items constant.
17. The computer-readable medium of claim 13 , wherein determining the content selection metric comprises: determining a count of the set of alternative net content values that exceed the net content value.
18. The computer-readable medium of claim 13 , wherein determining the content selection metric comprises: determining a percentage of the set of alternative net content values that exceed the net content value.
19. The computer-readable medium of claim 13 , wherein the content value offer represents an amount of real-world currency to be paid to the content distribution system.
20. The computer-readable medium of claim 13 , wherein the content selection metric is determined based on a plurality of net content values and a plurality of sets of alternative net content values, each net content value of the plurality of net content corresponding to a set of alternative net content values of the plurality of sets of alternative net content values.