Confidence intervals for key performance indicators in communication networks

1. A method, comprising:

performing, using one or more computer systems,

identifying vectors representing network events observed by a network monitoring system, each vector including: a dimension, an indication of a sampling ratio with which a respective network event was observed, and a value associated with the dimension;

calculating a Key Performance Indicator (KPI) corresponding to the observed network events for the dimension based, at least in part, upon the values; and

calculating a confidence associated with the KPI, based, at least in part, upon the sampling ratios, wherein calculating the confidence further includes:

performing, using the one or more computer systems,

estimating a number of network events (KN) that would have been observed by the network monitoring system in the absence of sampling;

determining a number of observed network events (Kn); and

calculating a standard deviation (Kσ) of the values.

2. The method of claim 1 , wherein the dimension includes at least one of: International Mobile Equipment Identity (IMEI), International Mobile Subscriber Identity (IMSI), Mobile Station Integrated Services Digital Network (MSISDN), User Agent (UA) Profile, User Agent, Handset Make, Handset Model, Software Version, Uniform Resource Locator (URL), Service, Application, Location, Mobile Country Code (MCC), or Mobile Network Code (MNC).

3. The method of claim 1 , wherein the dimension is observed with a first sampling ratio, wherein another dimension is observed with a second sampling ratio, and wherein the second sampling ratio is different from the first sampling ratio.

4. The method of claim 1 , wherein the sampling ratio includes an adaptive sampling ratio controlled by the network monitoring system in response to network or resource loading.

5. The method of claim 1 , wherein the sampling ratio varies over time.

6. The method of claim 1 , wherein the sampling ratio includes a whitelist or blacklist-based differentiated sampling ratio.

7. The method of claim 1 , wherein the value includes at least one of: session length, uplink byte count, downlink byte count, number of attempts, number of failures, or latency.

8. The method of claim 1 , wherein the confidence includes at least one of: a confidence level or a confidence interval, and wherein the confidence level or the confidence interval is based, at least in part, upon KN, Kn, and Kσ.

9. The method of claim 8 , wherein calculating the confidence further comprises:

performing, using the one or more computer systems,

identifying a desired confidence level; and

calculating a confidence interval based, at least in part, upon the desired confidence level.

10. The method of claim 8 , wherein calculating the confidence further comprises:

performing, using the one or more computer systems,

identifying a desired confidence interval; and

calculating a confidence level based, at least in part, upon the desired confidence interval.

11. The method of claim 8 , further comprising:

performing, using the one or more computer systems, causing a modification of a sampling ratio with which network events are to be observed in the future based, at least in part, upon a user's adjustment to the confidence level or to the confidence interval.

12. A network monitoring system, comprising:

a processor; and

a memory coupled to the processor, the memory configured to store program instructions executable by the processor to cause the network monitoring system to:

identify a plurality of vectors representing sampled network events, each vector including a plurality of dimensions, each of the plurality of dimensions associated with an indication of a sampling ratio with which the respective dimension was sampled, and a value associated with each respective dimension, the sampling ratios being subject to modification over time as a function of network or resource loading;

calculate a Key Performance Indicator (KPI) associated with a selected one of the plurality of dimensions based, at least in part, upon an operation performed with respect to respective values, the operation including at least one of a: mean, average, minimum, or maximum;

estimate a number of network events (KN) that would have been observed in the absence of sampling;

determine a number of sampled network events (Kn);

calculate a standard deviation (Kσ) of the values;

calculate a confidence value associated with the KPI, based, at least in part, upon KN, Kn, and Kσ;

display the KPI to a user; and

display the confidence value to the user in a graphical or textual format.

13. The network monitoring system of claim 12 , wherein to calculate the confidence value, the program instructions are further executable by the processor to cause the network monitoring system to:

identify a desired confidence level; and

calculate a confidence interval based, at least in part, upon the desired confidence level.

14. The network monitoring system of claim 12 , wherein to calculate the confidence value, the program instructions are further executable by the processor to cause the network monitoring system to:

identify a desired confidence interval; and

calculate a confidence level based, at least in part, upon the desired confidence interval.

15. The network monitoring system of claim 12 , the program instructions further executable by the processor to cause the network monitoring system to automatically modify a sampling ratio with which the selected dimension is to be sampled in subsequent network events based, at least in part, upon an adjustment to the confidence value.