SLA compliance determination with real user monitoring
Compliance with a Service Level Agreement (SLA) between an ecommerce provider and its customer may be determined using real user monitoring by collecting data for each HTTP/HTTPS request and response sent to the ecommerce provider. The presence of a content delivery network (CDN), also monitoring every HTTP/HTTPS request sent from a user, through the CDN provides an external, third-party confirmation of compliance. Real user monitoring is collected at the SaaS system by triggering an event rule, collecting data regarding the performance of the request and the response, and logging the created message. The log is processed and stored in a performance monitoring system where indexed for calculations and presentation.
1. A method of determining a service level for an internet-based Software as a Service comprising:
capturing a plurality of actual HTTP/HTTPS requests;
logging the plurality of HTTP/HTTPS requests;
capturing a plurality of actual responses related to the logged plurality of requests; and
logging the plurality of responses for a website hosted by a Software as a Service system; and
determining a service level request value for the website hosted by the Software as a Service system from the logged plurality of logged requests and the plurality of corresponding logged responses, the service level request value including a percentage for website availability.
2. The method of claim 1 wherein capturing a plurality of actual HTTP/HTTPS requests includes triggering an event at the Software as a Service system upon an actual http/https request.
3. The method of claim 1 wherein the capturing of a plurality of actual HTTP/HTTPS includes triggering an event upon an actual HTTP/HTTPS request, the event trigger being a data collection rule that fires upon an HTTP/HTTPS request.
4. The method of claim 1 wherein logging the plurality of HTTP/HTTP requests includes adding a time stamp to the request.
5. The method of claim 1 wherein determining a service level request value from the logged plurality of logged requests and the plurality of corresponding logged responses includes determining at least one metric indicative of the level of service for the internet-based Software as a service.
6. The method of claim 1 further comprising comparing the determined service level request value obtained from a plurality of actual HTTP/HTTPS requests to a service level request value produced by a synthetic monitoring of the Software as a Service.
7. The method of claim 1 further comprising combining the determined service level request value obtained from a plurality of actual HTTP/HTTPS requests and a service level request value produced by a synthetic monitoring of the Software as a Service to yield a third service level request value.
8. The method of determining a service level for an internet-based Software as a Service of claim 1 wherein the percentage for website availability is assessed substantially continually using real user traffic in near real time.
9. A computer system for determining a service level for an internet-based Software as a Service comprising:
an event monitor module associated with a Software as a Service system that
receives a first event input for a HTTP/HTTPS request at a Content Delivery Network; and
receives a second event input for an HTTP/HTTPS response from a Content Delivery Network;
a message log that receives a message for a first event input and receives a message for a second event input;
a logging module for parsing the message for the first event input and for parsing the message for the second event input;
a performance monitoring module associated with the Software as a Service system which includes an analytics module, the performance monitoring module determining an actual service level performance value for the Software as a Service system from parsed information from the logging module associated with the first event input and the second event input,
the computer system including computer hardware and software, the logging module, the performance monitoring module, and the analytics module including a combination of hardware and software.
10. The computer system for determining a service level of an internet-based Software as a Service of claim 9 wherein the first event input and the second event input are received from a Content Delivery Network and used to determine an actual service level to compare with a Service Level agreement.
11. The computer system for determining a service level of an Internet-based Software as a Service of claim 9 wherein the Software as a Service provider determines a synthetic service level performance value via synthetic monitoring, the synthetic service level performance value compared to the actual service level performance value.
12. The computer system for determining a service level of an Internet-based Software as a Service of claim 11 wherein the determination of the actual service level performance value is substantially independent of the determination of the synthetic service level performance value.
13. The computer system for determining a service level of an Internet-based Software as a Service of claim 9 wherein the Software as a Service provider determines a synthetic service level performance value via synthetic monitoring, the synthetic service level performance value combined with the actual service level performance value to produce an overall service level value.
14. The computer system of claim 9 wherein the actual service level of performance is calculable in real time.
15. The computer system of claim 9 wherein the actual service level value is calculated over a selected period of time, the selected period of time being less than a period of time in which a customer of the Software as a Service determines a similar service level value.
16. The computer system of claim 9 wherein the first message and the second message are in a first format and changed to a second format in the logging module.
17. The computer system of claim 9 wherein the performance monitoring module also outputs metrics associated with the system other than the service level value.
18. The computer system of claim 17 wherein the actual service level performance value is used to correct service levels when the service level performance drops below a selected level.
19. The computer system for determining a service level for an internet-based Software as a Service of claim 9 wherein the actual service level performance value includes a percentage for website availability.
20. A method of determining a service level for at least one website comprising:
using a Software as a Service system to host the at least one website, the system including a Content Delivery Network;
collecting request data from the Content Delivery Network;
collecting response data associated with the request data from the Content Delivery Network; and
determining a service level for the website hosted by the Software as a Service system using the collected request data and the collected response data.