Threshold selection for KPI candidacy in root cause analysis of network issues
In one embodiment, a network assurance service that monitors a network maps time series of values of key performance indicator (KPIs) measured from the network to lists of unique values from the time series. The service sets a target alarm rate for anomaly detection alarms raised by the network assurance service. The service uses an optimization function to identify a set of thresholds for the KPIs. The optimization function is based on: a comparison between the target alarm rate and a fraction of network issues flagged by the service as outliers, KPI thresholds selected based on the lists of unique values from the time series, and a number of thresholds that the KPIs must cross for the service to raise an alarm. The service raises an anomaly detection alarm for the monitored network based on the identified set of thresholds for the KPIs.
1. A method comprising:
mapping, by a network assurance service that monitors a network, time series of values of key performance indicator (KPIs) measured from the network to lists of unique values from the time series;
setting, by the network assurance service, a target alarm rate for anomaly detection alarms raised by the network assurance service;
using, by the network assurance service, an optimization function to identify a set of thresholds for the KPIs, wherein the optimization function is based on: a comparison between the target alarm rate and a fraction of network issues flagged by the network assurance service as outliers, KPI thresholds selected based on the lists of unique values from the time series, and a number of thresholds that the KPIs must cross for the network assurance service to raise an alarm; and
raising, by the network assurance service, an anomaly detection alarm for the monitored network based on the identified set of thresholds for the KPIs.
2. The method as in claim 1 , wherein setting the target alarm rate comprises:
receiving, at the network assurance service, user feedback regarding anomaly detection alarms raised by the network assurance service; and
selecting the target alarm rate based on the received user feedback.
3. The method as in claim 1 , wherein the target alarm rate represents a percentage of raised alarms or a number of raised alarms in a predefined time period.
4. The method as in claim 1 , wherein the KPIs comprise one or more of: packet failure rate, received signal strength indicator (RSSI), signal to noise ratio (SNR), or ratios of clients in the monitored network with low RSSI measurements.
5. The method as in claim 1 , wherein using the optimization function to identify a set of thresholds for the KPIs comprises:
applying a grid search, Nelder-Mead optimization, or L-BFGS optimization to the optimization function.
6. The method as in claim 1 , wherein the optimization function includes at least one hardcoded threshold.
7. The method as in claim 1 , wherein the network assurance service is a cloud-based service that is remote from the monitored network.
8. The method as in claim 1 , wherein the optimization function favors the KPI thresholds that are close to one another.
9. An apparatus, comprising:
one or more network interfaces to communicate with a network;
a processor coupled to the network interfaces and configured to execute one or more processes; and
a memory configured to store a process executable by the processor, the process when executed configured to:
map series of values of key performance indicator (KPIs) measured from a monitored network to lists of unique values from the time series;
set a target alarm rate for anomaly detection alarms raised by the network assurance service;
use an optimization function to identify a set of thresholds for the KPIs, wherein the optimization function is based on: a comparison between the target alarm rate and a fraction of network issues flagged by the apparatus as outliers, KPI thresholds selected based on the lists of unique values from the time series, and a number of thresholds that the KPIs must cross for the apparatus to raise an alarm; and
raise an anomaly detection alarm for the monitored network based on the identified set of thresholds for the KPIs.
10. The apparatus as in claim 9 , wherein the apparatus sets the target alarm rate by:
receiving user feedback regarding anomaly detection alarms raised by the network assurance service; and
selecting the target alarm rate based on the received user feedback.
11. The apparatus as in claim 9 , wherein the target alarm rate represents a percentage of raised alarms or a number of raised alarms in a predefined time period.
12. The apparatus as in claim 9 , wherein the KPIs comprise one or more of: packet failure rate, received signal strength indicator (RSSI), signal to noise ratio (SNR), or ratios of clients in the monitored network with low RSSI measurements.
13. The apparatus as in claim 9 , wherein the apparatus uses the optimization function to identify a set of thresholds for the KPIs by:
applying a grid search, Nelder-Mead optimization, or L-BFGS optimization to the optimization function.
14. The apparatus as in claim 9 , wherein the optimization function includes at least one hardcoded threshold.
15. The apparatus as in claim 9 , wherein the network assurance service is a cloud-based service that is remote from the monitored network.
16. The apparatus as in claim 9 , wherein the optimization function favors the KPI thresholds that are close to one another.
17. A tangible, non-transitory, computer-readable medium storing program instructions that cause a network assurance service that monitors a plurality of networks to execute a process comprising:
mapping, by the network assurance service that monitors the network, time series of values of key performance indicator (KPIs) measured from the network to lists of unique values from the time series;
setting, by the network assurance service, a target alarm rate for anomaly detection alarms raised by the network assurance service;
using, by the network assurance service, an optimization function to identify a set of thresholds for the KPIs, wherein the optimization function is based on: a comparison between the target alarm rate and a fraction of network issues flagged by the network assurance service as outliers, KPI thresholds selected based on the lists of unique values from the time series, and a number of thresholds that the KPIs must cross for the network assurance service to raise an alarm; and
raising, by the network assurance service, an anomaly detection alarm for the monitored network based on the identified set of thresholds for the KPIs.
18. The computer-readable medium as in claim 17 , wherein setting the target alarm rate comprises:
receiving, at the network assurance service, user feedback regarding anomaly detection alarms raised by the network assurance service; and
selecting the target alarm rate based on the received user feedback.
19. The computer-readable medium as in claim 17 , wherein the KPIs comprise one or more of: packet failure rate, received signal strength indicator (RSSI), signal to noise ratio (SNR), or ratios of clients in the monitored network with low RSSI measurements.
20. The computer-readable medium as in claim 17 , wherein using the optimization function to identify a set of thresholds for the KPIs comprises:
applying a grid search, Nelder-Mead optimization, or L-BFGS optimization to the optimization function.