Using machine learning to improve detection of visual pairwise differences between browsers

1. A method comprising, by one or more processors associated with one or more computer systems:

receiving information associated with a first plurality of matched image pairs;

assessing each matched image pair in the first plurality of matched image pairs to detect a first set of visual pairwise differences that qualify as an error, wherein a visual pairwise difference comprises a difference with respect to at least one of position, size, color, or style information from Cascading Style Sheets (CSS);

training, by one or more of the processors, a prediction engine based upon the first plurality of matched image pairs and the first set of visual pairwise differences using a machine learning training module, wherein the prediction engine learns to classify visual pairwise differences based on a decision tree classifier that incorporates one or more algorithms to compute one or more of the differences;

receiving information associated with a second plurality of matched image pairs, wherein each matched image pair in the second plurality of matched image pairs comprises at least a portion of a visual end-user experience screen of an event-driven application executed in a client-tier environment;

assessing, using the prediction engine, each matched image pair in the second plurality of matched image pairs to detect a second set of visual pairwise differences that qualify as an error;

receiving user feedback, for at least one matched image pair in the second plurality of matched image pairs that was assessed to have at least one visual pairwise difference from the second set of visual pairwise differences, that the assessed visual pairwise difference from the second set of visual pairwise differences should not have qualified as an error; and

re-training, by one or more of the processors, the prediction engine based on the user feedback.

2. The method of claim 1 , wherein one or more of the visual pairwise differences from the first set or second set of visual pairwise differences comprises a difference with respect to position, and wherein assessing an image pair to detect the visual pairwise difference comprises assessing, for each image in the image pair, information associated with at least one of: a displacement ratio, a size, a color difference ratio, a color density, or an Earth Movers' Distance (EMD).

3. The method of claim 1 , wherein one or more of the visual pairwise differences from the first set or second set of visual pairwise differences comprises a difference with respect to size, and wherein assessing an image pair to detect the visual pairwise difference comprises assessing, for each image in the image pair, information associated with at least one of: a size difference ratio, a size, a color density, or an Earth Movers' Distance (EMD).

4. The method of claim 1 , wherein one or more of the visual pairwise differences from the first set or second set of visual pairwise differences comprises a difference with respect to style, and wherein assessing an image pair to detect the visual pairwise difference comprises assessing, for each image in the image pair, information associated with at least one of: HyperText Markup Language (HTML) tags, a browser of the client-tier environment, a size, a color density, or an Earth Movers' Distance (EMD).

5. The method of claim 1 , further comprising assessing, using the prediction engine, each image pair in the second plurality of image pairs to detect a third set of visual pairwise differences that do not qualify as an error, wherein a visual pairwise difference that does not qualify as an error comprises a difference that is related to browser-specific rendering of the event-driven application.

6. The method of claim 1 , wherein the event-driven application is a web application.

7. The method of claim 1 , wherein each client-tier environment comprises:

a particular web browser;

a version of the particular web browser;

a configuration of the particular web browser;

a particular operating system (OS) of a computer system that the particular web browser is running on; and

a screen resolution of a display of the computer system.

8. One or more computer-readable non-transitory storage media embodying software that is executed to:

receive information associated with a first plurality of matched image pairs;

assess each matched image pair in the first plurality of matched image pairs to detect a first set of visual pairwise differences that qualify as an error, wherein a visual pairwise difference comprises a difference with respect to at least one of position, size, color, or style information from Cascading Style Sheets (CSS);

train a prediction engine based upon the first plurality of matched image pairs and the first set of visual pairwise differences using a machine learning training module, wherein the prediction engine learns to classify visual pairwise differences based on a decision tree classifier that incorporates one or more algorithms computing one or more of the differences;

receive information associated with a second plurality of matched image pairs, wherein each matched image pair in the second plurality of matched image pairs comprises at least a portion of a visual end-user experience screen of an event-driven application executed in a client-tier environment;

assess, using the prediction engine, each matched image pair in the second plurality of matched image pairs to detect a second set of visual pairwise differences that qualify as an error;

receive user feedback, for at least one matched image pair in the second plurality of matched image pairs that was assessed to have at least one visual pairwise difference from the second set of visual pairwise differences, that the assessed visual pairwise difference from the second set of visual pairwise differences should not have qualified as an error; and

re-train the prediction engine based on the user feedback.

9. The media of claim 8 , wherein one or more of the visual pairwise differences from the first set or second set of visual pairwise differences comprises a difference with respect to position, and wherein assessing an image pair to detect the visual pairwise difference comprises assessing, for each image in the image pair, information associated with at least one of: a displacement ratio, a size, a color difference ratio, a color density, or an Earth Movers' Distance (EMD).

10. The media of claim 8 , wherein one or more of the visual pairwise differences from the first set or second set of visual pairwise differences comprises a difference with respect to size, and wherein assessing an image pair to detect the visual pairwise difference comprises assessing, for each image in the image pair, information associated with at least one of: a size difference ratio, a size, a color density, or an Earth Movers' Distance (EMD).

11. The media of claim 8 , wherein one or more of the visual pairwise differences from the first set or second set of visual pairwise differences comprises a difference with respect to style, and wherein assessing an image pair to detect the visual pairwise difference comprises assessing, for each image in the image pair, information associated with at least one of: HyperText Markup Language (HTML) tags, a browser of the client-tier environment, a size, a color density, or an Earth Movers' Distance (EMD).

12. The media of claim 8 , wherein the software is further operable when executed to assess, using the prediction engine, each image pair in the second plurality of image pairs to detect a third set of visual pairwise differences that do not qualify as an error, wherein a visual pairwise difference that does not qualify as an error comprises a difference that is related to browser-specific rendering of the event-driven application.

13. The media of claim 8 , wherein the event-driven application is a web application.

14. The media of claim 8 , wherein each client-tier environment comprises:

a particular web browser;

a version of the particular web browser;

a configuration of the particular web browser;

a particular operating system (OS) of a computer system that the particular web browser is running on; and

a screen resolution of a display of the computer system.

15. A system comprising: one or more processors;

and a memory coupled to the one or more processors comprising instructions executable by the one or more processors, the one or more processors executing the instructions to:

receive information associated with a first plurality of matched image pairs;

assess each matched image pair in the first plurality of matched image pairs to detect a first set of visual pairwise differences that qualify as an error, wherein a visual pairwise difference comprises a difference with respect to at least one of position, size, color, or style information from Cascading Style Sheets (CSS);

train a prediction engine based upon the first plurality of matched image pairs and the first set of visual pairwise differences using a machine learning training module, wherein the prediction engine learns to classify visual pairwise differences based on a decision tree classifier that incorporates one or more algorithms computing one or more of the differences;

receive information associated with a second plurality of matched image pairs, wherein each matched image pair in the second plurality of matched image pairs comprises at least a portion of a visual end-user experience screen of an event-driven application executed in a client-tier environment;

assess, using the prediction engine, each matched image pair in the second plurality of matched image pairs to detect a second set of visual pairwise differences that qualify as an error;

receive user feedback, for at least one matched image pair in the second plurality of matched image pairs that was assessed to have at least one visual pairwise difference from the second set of visual pairwise differences, that the assessed visual pairwise difference from the second set of visual pairwise differences should not have qualified as an error; and

re-train the prediction engine based on the user feedback.

16. The system of claim 15 , wherein one or more of the visual pairwise differences from the first set or second set of visual pairwise differences comprises a difference with respect to position, and wherein assessing an image pair to detect the visual pairwise difference comprises assessing, for each image in the image pair, information associated with at least one of: a displacement ratio, a size, a color difference ratio, a color density, or an Earth Movers' Distance (EMD).

17. The system of claim 15 , wherein one or more of the visual pairwise differences from the first set or second set of visual pairwise differences comprises a difference with respect to size, and wherein assessing an image pair to detect the visual pairwise difference comprises assessing, for each image in the image pair, information associated with at least one of: a size difference ratio, a size, a color density, or an Earth Movers' Distance (EMD).

18. The system of claim 15 , wherein one or more of the visual pairwise differences from the first set or second set of visual pairwise differences comprises a difference with respect to style, and wherein assessing an image pair to detect the visual pairwise difference comprises assessing, for each image in the image pair, information associated with at least one of: HyperText Markup Language (HTML) tags, a browser of the client-tier environment, a size, a color density, or an Earth Movers' Distance (EMD).

19. The system of claim 15 , wherein the system is further operable when executing the instructions to assess, using the prediction engine, each image pair in the second plurality of image pairs to detect a third set of visual pairwise differences that do not qualify as an error, wherein a visual pairwise difference that does not qualify as an error comprises a difference that is related to browser-specific rendering of the event-driven application.

20. The system of claim 15 , wherein the event-driven application is a web application.