Quality of experience policy engine for custom applications based on feedback
In one embodiment, a method herein may comprise: causing, for a quality-of-experience evaluation session, one or more network impairments to be injected according to a set of predefined scenarios on application traffic for a plurality of feedback sources that are using a particular application in a computer network; obtaining experience-based feedback from the plurality of feedback sources for the quality-of-experience evaluation session; correlating the experience-based feedback with the one or more network impairments to produce an evaluation result for the quality-of-experience evaluation session; and generating a quality-of-experience-based network policy recommendation for the particular application based on the evaluation result.
1 . A method, comprising:
redirecting live, actual application traffic from users within an enterprise environment to one or more network proxies;
causing, by a device and for a quality-of-experience evaluation session, one or more network impairments to be injected into the one or more network proxies according to a set of predefined scenarios on the live, actual application traffic for a plurality of feedback sources that are using a particular application in a computer network within the enterprise environment, wherein the feedback sources comprise the users;
obtaining, by the device, explicit, user-provided experience-based feedback in response to prompts issued in real-time after each of the set of predefined scenarios executed during the quality-of-experience evaluation session;
correlating, by the device, each item of the explicit, user-provided experience-based feedback with a corresponding predefined impairment scenario and its impairment parameters to produce an evaluation result comprising application-specific service level agreement threshold values for the quality-of-experience evaluation session; and
generating, by the device, a quality-of-experience-based network policy recommendation for the particular application based on the evaluation result.
2 . The method of claim 1 , wherein the one or more network impairments to be injected according to the set of predefined scenarios comprise one or more impairment patterns selected from a group consisting of: a single isolated metric impairment pattern; an impairment pattern for a combination of a plurality of metrics; a continuous time distribution pattern; a seasonal time distribution pattern; and a random time distribution pattern.
3 . The method of claim 1 , further comprising:
measuring unaltered network conditions for the particular application in the computer network prior to the quality-of-experience evaluation session; and
determining, based on the unaltered network conditions prior to the quality-of-experience evaluation session, the one or more network impairments to be injected in order to meet the set of predefined scenarios.
4 . The method of claim 1 , further comprising:
performing a subsequent information inquiry with one or more particular feedback sources of the plurality of feedback sources responsive to comparatively low experience-based feedback for the quality-of-experience evaluation session from the one or more particular feedback sources.
5 . The method of claim 1 , further comprising:
selecting a set of users that have historical usage of the particular application for the plurality of feedback sources.
6 . The method of claim 1 , further comprising:
notifying a set of candidate users about the quality-of-experience evaluation session; and
selecting a particular set of participant users from the set of candidate users that have agreed to participate in the quality-of-experience evaluation session as the plurality of feedback sources.
7 . The method of claim 1 , wherein redirecting comprises:
redirecting the live, actual application traffic for the plurality of feedback sources using the particular application via one or more network devices configured to inject the one or more network impairments according to the set of predefined scenarios on the actual application traffic.
8 . The method of claim 7 , further comprising:
determining where within the computer network to configure the one or more network devices based on one or more factors selected from: manual configuration; dynamic configuration based on first locations with feedback indicating a poor quality-of-experience measure for the particular application; and dynamic configuration based on second locations raising network alarms without feedback indications of a poor quality-of-experience measure.
9 . The method of claim 1 , wherein the set of predefined scenarios comprises increased impairment intervals that are executed until reaching a given number of consecutive feedback indications of a poor quality-of-experience measure.
10 . The method of claim 9 , wherein the set of predefined scenarios further comprises a subsequent set of increased impairment intervals that start at a higher level of impairment than a previous set of increased impairment intervals based on when the previous set of increased impairment intervals reached the given number of consecutive feedback indications of the poor quality-of-experience measure.
11 . The method of claim 1 , wherein generating comprises:
defining application-specific service level agreement threshold values.
12 . The method of claim 11 , further comprising:
using a machine learning model to find extrema for the application-specific service level agreement threshold values that still achieve a target quality-of-experience measure for the particular application with high probability based on the evaluation result.
13 . The method of claim 12 , further comprising:
adding contextual information to the machine learning model selected from a group consisting of: date and time; site-level router and link metrics; a type of task undertaken by the plurality of feedback sources; and actions carried out by the plurality of feedback sources.
14 . The method of claim 1 , further comprising:
optimizing the quality-of-experience-based network policy recommendation for the particular application against one or more defined objectives in addition to quality-of-experience.
15 . The method of claim 1 , wherein the quality-of-experience evaluation session is region-specific, and wherein the quality-of-experience-based network policy recommendation for the particular application is region-specific.
16 . The method of claim 1 , wherein the one or more network impairments to be injected according to the set of predefined scenarios are based on one or more constraints selected from a group consisting of: one or more particular applications to evaluate; duration of the quality-of-experience evaluation session; frequency of the quality-of-experience evaluation session; a number of feedback sources for the plurality of feedback sources; a time of day for the quality-of-experience evaluation session; a geographical location for the quality-of-experience evaluation session; one or more specific sites that should be excluded from the quality-of-experience evaluation session; a minimum desired quality-of-experience measure; and one or more objectives for policy optimization.
17 . An apparatus, comprising:
one or more network interfaces to communicate with a network;
a processor coupled to the one or more network interfaces and configured to execute one or more processes; and
a memory configured to store a process that is executable by the processor, the process comprising:
redirecting live, actual application traffic from users within an enterprise environment to one or more network proxies;
causing, by a device and for a quality-of-experience evaluation session, one or more network impairments to be injected into the one or more network proxies according to a set of predefined scenarios on the live, actual application traffic for a plurality of feedback sources that are using a particular application in a computer network within the enterprise environment, wherein the feedback sources comprise the users;
obtaining, by the device, explicit, user-provided experience-based feedback in response to prompts issued in real-time after each of the set of predefined scenarios executed during the quality-of-experience evaluation session;
correlating, by the device, each item of the explicit, user-provided experience-based feedback with a corresponding predefined impairment scenario and its impairment parameters to produce an evaluation result comprising application-specific service level agreement threshold values for the quality-of-experience evaluation session; and
generating, by the device, a quality-of-experience-based network policy recommendation for the particular application based on the evaluation result.
18 . The apparatus of claim 17 , the process further comprising:
measuring unaltered network conditions for the particular application in the computer network prior to the quality-of-experience evaluation session; and
determining, based on the unaltered network conditions prior to the quality-of-experience evaluation session, the one or more network impairments to be injected in order to meet the set of predefined scenarios.
19 . The apparatus of claim 17 , wherein causing comprises:
redirecting the application traffic for the plurality of feedback sources using the particular application via one or more network devices configured to inject the one or more network impairments according to the set of predefined scenarios on the application traffic.
20 . A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:
redirecting live, actual application traffic from users within an enterprise environment to one or more network proxies;
causing, by a device and for a quality-of-experience evaluation session, one or more network impairments to be injected into the one or more network proxies according to a set of predefined scenarios on the live, actual application traffic for a plurality of feedback sources that are using a particular application in a computer network within the enterprise environment, wherein the feedback sources comprise the users;
obtaining, by the device, explicit, user-provided experience-based feedback in response to prompts issued in real-time after each of the set of predefined scenarios executed during the quality-of-experience evaluation session;
correlating, by the device, each item of the explicit, user-provided experience-based feedback with a corresponding predefined impairment scenario and its impairment parameters to produce an evaluation result comprising application-specific service level agreement threshold values for the quality-of-experience evaluation session; and
generating, by the device, a quality-of-experience-based network policy recommendation for the particular application based on the evaluation result.