Measuring mobile application program reliability across multiple operating systems
A quality score for a computer application release is determined using a first number of unique users who have launched the computer application release on user devices and a second number of unique users who have encountered at least once an abnormal termination with the computer application release on user devices. Additionally or optionally, an application quality score can be computed for a computer application based on quality scores of computer application releases that represent different versions of the computer application. Additionally or optionally, a weighted application quality score can be computed for a computer application by further taking into consideration the average application quality score and popularity of a plurality of computer applications.
1. A method, comprising:
receiving, by a server, application performance events from monitoring code operating on a plurality of mobile devices, the monitoring code included in a computer application program and monitors operations of the computer application program executing on a mobile device, the monitoring code sends application performance events relating to the computer application program to the server;
calculating, by the server, a user experience score for each of two or more operating systems running on the plurality of mobile devices using application performance data extracted from received application performance events associated with a particular computer application program;
causing display, by the server, of the user experience score or a representation of the user experience score for each of the two or more operating systems.
2. The method as recited in claim 1 , wherein the application performance events include application performance data relating to the computer application program prior to an error occurring in the computer application program.
3. The method as recited in claim 1 , further comprising:
classifying, by the server, the received application performance events into a plurality of specific types of errors by evaluating the application performance data in each application performance event of the received application performance events;
calculating, by the server, a number of unique occurrences of each specific type of error among the plurality of specific types of errors associated with the particular computer application program, the number of unique occurrences of each specific type of error indicating a number of unique users that have launched the particular computer application program and have been affected by the specific type of error;
wherein the causing display further comprises:
causing display of information relating to a number of unique occurrences of a particular type of error associated with the particular computer application program for each of the two or more operating systems.
4. The method as recited in claim 1 ,
classifying, by the server, the received application performance events into a plurality of specific types of errors by evaluating the application performance data in each application performance event of the received application performance events;
calculating, by the server, a number of unique occurrences of each specific type of error among the plurality of specific types of errors associated with a plurality of versions of the particular computer application program;
wherein the causing display further comprises:
causing display of information relating to the number of unique occurrences of the particular type of error associated with the plurality of versions of the particular computer application program for each of the two or more operating systems.
5. The method as recited in claim 1 , further comprising:
classifying, by the server, the received application performance events into a plurality of specific types of errors by evaluating the application performance data in each application performance event of the received application performance events;
calculating, by the server, a number of unique occurrences of each specific type of error among the plurality of specific types of errors associated with a plurality of versions of the particular computer application program;
wherein the causing display further comprises:
causing display of information relating to the number of unique occurrences of the particular type of error associated with the plurality of versions of the particular computer application program along with an associated version number for each of the two or more operating systems.
6. The method as recited in claim 1 , wherein the calculating the user experience score calculates the user experience score for a specific time period.
7. The method as recited in claim 1 , wherein the calculating the user experience score calculates the user experience score for a particular geographical area for mobile devices running the particular computer application program.
8. The method as recited in claim 1 , wherein the calculating the user experience score calculates the user experience score for a particular mobile device type running the particular computer application program.
9. The method as recited in claim 1 , wherein at least one application performance event among the application performance events includes a trace leading up to an error.
10. The method as recited in claim 1 , wherein at least one application performance event among the application performance events includes information about system conditions under which the application performance event occurred.
11. The method as recited in claim 1 , further comprising:
retrieving a set of user experience scores for the particular computer application program over a series of version numbers from a datastore; and
wherein the causing display further comprises:
causing display of each user experience score in the set of user experience scores along with an associated version number.
12. The method as recited in claim 1 , wherein the calculating the user experience score includes a popularity rating of the particular computer application program in relation to other computer application programs.
13. One or more non-transitory computer-readable media, storing one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform:
receiving, by a server, application performance events from monitoring code operating on a plurality of mobile devices, the monitoring code included in a computer application program and monitors operations of the computer application program executing on a mobile device, the monitoring code sends application performance events relating to the computer application program to the server;
calculating, by the server, a user experience score for each of two or more operating systems running on the plurality of mobile devices using application performance data extracted from received application performance events associated with a particular computer application program;
causing display, by the server, of the user experience score or a representation of the user experience score for each of the two or more operating systems.
14. The one or more non-transitory computer-readable media as recited in claim 13 , wherein the application performance events include application performance data relating to the computer application program prior to an error occurring in the computer application program.
15. The one or more non-transitory computer-readable media as recited in claim 13 , further comprising:
classifying, by the server, the received application performance events into a plurality of specific types of errors by evaluating the application performance data in each application performance event of the received application performance events;
calculating, by the server, a number of unique occurrences of each specific type of error among the plurality of specific types of errors associated with a particular computer application program, the number of unique occurrences of each specific type of error indicating a number of unique users that have launched the particular computer application program and have been affected by the specific type of error;
wherein the causing display further comprises:
causing display of information relating to a number of unique occurrences of a particular type of error associated with the particular computer application program for each of the two or more operating systems.
16. The one or more non-transitory computer-readable media as recited in claim 13 , wherein the calculating the user experience score calculates the user experience score for a particular geographical area for mobile devices running the particular computer application program.
17. A system, comprising:
an application performance event receiving device, implemented at least partially in hardware at a server, that receives application performance events from monitoring code operating on a plurality of mobile devices, the monitoring code included in a computer application program and monitors operations of the computer application program executing on a mobile device, the monitoring code sends application performance events relating to the computer application program to the server;
a user experience calculation device, implemented at least partially in hardware at the server, that calculates a user experience score for each of two or more operating systems running on the plurality of mobile devices using application performance data extracted from received application performance events associated with a particular computer application program;
a display formatting device, implemented at least partially in hardware at the server, that causes display of the user experience score or a representation of the user experience score for each of the two or more operating systems.
18. The system as recited in claim 17 , wherein the application performance events include application performance data relating to the computer application program prior to an error occurring in the computer application program.
19. The system as recited in claim 17 , further comprising:
an event classification device, implemented at least partially in hardware at the server, that classifies the received application performance events into a plurality of specific types of errors by evaluating the application performance data in each application performance event of the received application performance events;
an error occurrence calculation device, implemented at least partially in hardware at the server, that calculates a number of unique occurrences of each specific type of error among the plurality of specific types of errors associated with a particular computer application program, the number of unique occurrences of each specific type of error indicating a number of unique users that have launched the particular computer application program and have been affected by the specific type of error;
wherein the display formatting device causes display of information relating to a number of unique occurrences of a particular type of error associated with the particular computer application program for each of the two or more operating systems.
20. The system as recited in claim 17 , wherein the user experience calculation device calculates the user experience score for a particular geographical area for mobile devices running the particular computer application program.