Evaluating content for compliance with a content policy enforced by an online system using a machine learning model determining compliance with another content policy

1. A method comprising:

maintaining one or more machine learning models at an online system, each machine learning model associated with a content policy and generating a risk score for one or more content items received by the online system for display to users of the online system, the risk score indicating a likelihood of a content item violating the content policy, each maintained machine learning model trained based on the associated content policy and comprising at least one of: neural networks, deep learning models, and support vector machines;

obtaining an additional content policy differing from the content policies associated with the one or more maintained machine learning models;

selecting a maintained machine learning model associated with a particular content policy;

obtaining a set of content items including content items identified as violating the additional content policy and content items identified as not violating the additional content policy;

determining a set of risk scores by applying the selected maintained machine learning model to each content item of the set of content items;

generating a mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the additional content policy;

storing the generated mapping in association with the additional content policy;

determining risk scores for other content items by applying the selected maintained machine learning model to each of the other content items;

determining likelihoods of each of the other content items violating the additional content policy by mapping determined risk scores for other content items to the risk scores determined for the other content items based on the stored mapping;

ranking the other content items based on the determined likelihoods by:

determining an expected number of impressions of each of the other content items based on characteristics of content items and characteristics of users for whom the online system identified opportunities to present content;

for each of the other content items, determining a product of the expected number of impressions of a content item and a likelihood of the content item violating the additional content policy; and

ranking the other content items based on the determined products; and

queuing the other content items for manual review for compliance with the additional content policy based on the ranking.

2. The method of claim 1 , further comprising:

presenting the other content items to one or more human reviewers to evaluate for compliance with the additional content policy in an order specified by the queue.

3. The method of claim 1 , wherein generating the mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the additional content policy comprises:

identifying a plurality of ranges of risk scores;

for each of the plurality of ranges:

determining a total number of content items of the set having risk scores within a range;

determining a number of content items of the set identified as violating the additional content policy and having risk scores within the range; and

determining a ratio of the number of content items of the set identified as violating the additional content policy and having risk scores within the range to the total number of content items of the set having risk scores within the range; and

storing the determined ratio in association with the range.

4. The method of claim 3 , wherein generating the mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the additional content policy further comprises:

generating a function mapping risk scores from application of the selected maintained machine learning model to likelihoods of violating the additional content policy based on the stored associations between determined ratios and ranges.

5. The method of claim 3 , wherein generating the mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the additional content policy further comprises:

generating a table correlating ranges with likelihoods of violating the additional content policy.

6. A method comprising:

maintaining one or more machine learning models at an online system, each machine learning model associated with a content policy and generating a risk score for one or more content items received by the online system for display to users of the online system, the risk score indicating a likelihood of a content item violating the content policy, each maintained machine learning model trained based on the associated content policy and comprising at least one of: neural networks, deep learning models, and support vector machines;

obtaining an additional type of content item including one or more different characteristics than types of content items previously received by the online system;

selecting a maintained machine learning model associated with a particular content policy;

obtaining a set of content items having the additional type, the set of content items including content items identified as violating the particular content policy and content items identified as not violating the particular content policy;

determining a set of risk scores by applying the selected maintained machine learning model to each content item of the set of content items;

generating a mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the additional content policy;

storing the generated mapping in association with the additional type of content item;

determining risk scores for other content items having the additional type by applying the selected maintained machine learning model to each of the other content items having the additional type; and

determining likelihoods of each of the other content items having the additional type violating the particular content policy by mapping determined risk scores for other content items to the risk scores determined for the other content items having the additional type based on the stored mapping

ranking the other content items having the additional type based on the determined likelihoods by:

determining an expected number of impressions of each of the other content items having the additional type based on characteristics of content items having the additional type and characteristics of users for whom the online system identified opportunities to present content;

for each of the other content items having the additional type, determining a product u of impressions of a content item having the additional type and a likelihood of the content item having the additional type violating the additional content policy; and

ranking the other content items having the additional type based on the determined products; and

queuing the other content items having the additional type for manual review for compliance with the particular content policy based on the ranking.

7. The method of claim 6 , further comprising:

presenting the other content items having the additional type to one or more human reviewers to evaluate for compliance with the particular content policy in an order specified by the queue.

8. The method of claim 6 , wherein generating the mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the particular content policy comprises:

identifying a plurality of ranges of risk scores;

for each of the plurality of ranges:

determining a total number of content items of the set having risk scores within a range;

determining a number of content items of the set identified as violating the additional content policy and having risk scores within the range; and

determining a ratio of the number of content items of the set identified as violating the additional content policy and having risk scores within the range to the total number of content items of the set having risk scores within the range; and

storing the determined ratio in association with the range.

9. The method of claim 8 , wherein generating the mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the particular content policy further comprises:

generating a function mapping risk scores from application of the selected maintained machine learning model to likelihoods of violating the particular content policy based on the stored associations between determined ratios and ranges.

10. The method of claim 8 , wherein generating the mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the particular content policy further comprises:

generating a table correlating ranges with likelihoods of violating the particular content policy.

11. A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

maintain one or more machine learning models at an online system, each machine learning model associated with a content policy and generating a risk score for one or more content items received by the online system for display to users of the online system, the risk score indicating a likelihood of a content item violating the content policy, each maintained machine learning model trained based on the associated content policy and comprising at least one of: neural networks, deep learning models, and support vector machines;

obtain an additional content policy differing from the content policies associated with the one or more maintained machine learning models;

select a maintained machine learning model associated with a particular content policy;

obtain a set of content items including content items identified as violating the additional content policy and content items identified as not violating the additional content policy;

determine a set of risk scores by applying the selected maintained machine learning model to each content item of the set of content items;

generate a mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the additional content policy;

store the generated mapping in association with the additional content policy;

determine risk scores for other content items by applying the selected maintained machine learning model to each of the other content items; and

determine likelihoods of each of the other content items violating the additional content policy by mapping determined risk scores for other content items to the risk scores determined for the other content items based on the stored mapping;

rank the other content items based on the determined likelihoods by:

determining an expected number of impressions of each of the other content items based on characteristics of content items and characteristics of users for whom the online system identified opportunities to present content;

for each of the other content items, determining a product of the expected number of impressions of a content item and a likelihood of the content item violating the additional content policy; and

ranking the other content items based on the determined products; and

queue the other content items for manual review for compliance with the additional content policy based on the ranking.

12. The computer program product of claim 11 , wherein generate the mapping between risk scores determined by application of the selected maintained machine learning model and rates at which content items of the set violate the additional content policy comprises:

identify a plurality of ranges of risk scores;

for each of the plurality of ranges:

determine a total number of content items of the set having risk scores within a range;

determine a number of content items of the set identified as violating the additional content policy and having risk scores within the range; and

determine a ratio of the number of content items of the set identified as violating the additional content policy and having risk scores within the range to the total number of content items of the set having risk scores within the range; and

store the determined ratio in association with the range.