Ranking services and top N rank lists
Techniques for ranking services and top N rank lists are disclosed. In some embodiments, a system, process, and/or computer program product for ranking services and top N rank lists includes receiving a set of network related event data, wherein the set of network related event data includes Domain Name System (DNS) related event data; aggregating the DNS related event data over a period of time and rank order by popularity; and generating a top N rank list for ranking popularity over the period of time for a set of domains using the aggregated DNS related event data and rank order by popularity.
1 . A system, comprising:
a processor configured to:
receive a set of network related event data, wherein the set of network related event data includes Domain Name System (DNS) related event data;
aggregate the DNS related event data over a period of time and rank order by popularity;
generate a top N rank list for ranking popularity over the period of time for a set of domains using the aggregated DNS related event data and rank order by popularity; and
determine rank intervals using the top N rank list by aggregating the data over a set of consecutive periods of days to obtain ranks over the period of time for the set of domains, comprising to:
determine, over the period of time, a confidence interval for a rank of one domain of a plurality of domains of the set of network related event data, wherein the period of time includes a number of T days, wherein the determining of the confidence interval comprises to:
approximate daily T ranks associated with the one domain to a normal distribution, wherein the daily T ranks are determined based on ordered observation counts of the one domain; and
determine the confidence interval based on the normal distribution; and
order, based on popularity, the confidence interval of the one domain and confidence interval of another domain of the plurality of domains to obtain the rank intervals; and
output, based on the rank intervals, a rank interval list for the set of domains that is used by a network device to enforce a network security policy using the rank interval list; and
a memory coupled to the processor and configured to provide the processor with instructions.
2 . The system recited in claim 1 , wherein the processor is further configured to:
normalize the ranks; and
generate confidence intervals based on the normalized ranks.
3 . The system recited in claim 1 , wherein the processor is further configured to:
generate, based on the rank intervals, a most likely rank and an interval range for the set of domains.
4 . A method, comprising:
receiving a set of network related event data, wherein the set of network related event data includes Domain Name System (DNS) related event data;
aggregating the DNS related event data over a period of time and rank order by popularity; and
generating a top N rank list for ranking popularity over the period of time for a set of domains using the aggregated DNS related event data and rank order by popularity; and
determining rank intervals using the top N rank list by aggregating the data over a set of consecutive period of days to obtain ranks over the period of time for the set of domains, comprising:
determining, over the period of time, a confidence interval for a rank of one domain of a plurality of domains of the set of network related event data, wherein the period of time includes a number of T days, wherein the determining of the confidence interval comprises:
approximating daily T ranks associated with the one domain to a normal distribution, wherein the daily T ranks are determined based on ordered observation counts of the one domain; and
determining the confidence interval based on the normal distribution; and
ordering, based on popularity, the confidence interval of the one domain and confidence interval of another domain of the plurality of domains to obtain the rank intervals; and
outputting, based on the rank intervals, a rank interval list for the set of domains that is used by a network device to enforce a network security policy using the rank interval list.
5 . The method of claim 4 , further comprising:
normalizing the ranks; and
generating confidence intervals based on the normalized ranks.
6 . The method of claim 4 , further comprising:
generating, based on the rank intervals, a most likely rank and an interval range for the set of domains.
7 . A non-transitory computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving a set of network related event data, wherein the set of network related event data includes Domain Name System (DNS) related event data;
aggregating the DNS related event data over a period of time and rank order by popularity;
generating a top N rank list for ranking popularity over the period of time for a set of domains using the aggregated DNS related event data and rank order by popularity; and
determining rank intervals using the top N rank list by aggregating the data over a set of consecutive periods of days to obtain ranks over the period of time for the set of domains, comprising:
determining, over the period of time, a confidence interval for a rank of one domain of a plurality of domains of the set of network related event data, wherein the period of time includes a number of T days, wherein the determining of the confidence interval comprises:
approximating daily T ranks associated with the one domain to a normal distribution, wherein the daily T ranks are determined based on ordered observation counts of the one domain; and
determining the confidence interval based on the normal distribution; and
ordering, based on popularity, the confidence interval of the one domain and confidence interval of another domain of the plurality of domains to obtain the rank intervals; and
outputting, based on the rank intervals, a rank interval list for the set of domains that is used by a network device to enforce a network security policy using the rank interval list.
8 . The non-transitory computer program product recited in claim 7 , further comprising computer instructions for:
normalizing the ranks; and
generating the confidence intervals based on the normalized ranks.
9 . The non-transitory computer program product recited in claim 7 , further comprising computer instructions for:
generating, based on the rank intervals, a most likely rank and an interval range for the set of domains.