Testing the rendering of visual content

1. A method performed by a data processing apparatus, the method comprising:

receiving input specifying a change to a parameter relating to display of a visual content in a webpage;

in response to receiving the input, generating a test image of the webpage including the visual content, wherein the test image of the webpage represents a visual appearance of the webpage as displayed to a user of a client device;

comparing the generated test image with a predetermined target image of the visual content to determine whether the visual content included in the generated test image is visually consistent with the predetermined target image of the visual content, wherein the predetermined target image of the visual content represents an expected visual appearance of the visual content as displayed to the user of the client device, and the predetermined target image is stored in a repository; and

selectively providing a failure notification based upon a result of the comparison;

wherein comparing the generated test image with the predetermined target image of the visual content comprises:

selecting a plurality of different portions of the generated test image, where each portion has a same size as the predetermined target image,

determining, for each portion, a difference for the portion by comparing pixels of the portion with pixels of the predetermined target image,

identifying a portion of the plurality of different portions having a smallest difference, and

determining whether the smallest difference of the portion is smaller than a threshold difference;

wherein the visual content included in the generated test image of the webpage is determined to be visually consistent with the predetermined target image of the visual content when the smallest difference for the portion is smaller than the threshold difference.

2. The method of claim 1 , wherein the visual content comprises an advertisement.

3. The method of claim 1 , wherein receiving input specifying the change to the parameter comprises:

receiving input specifying a change to one or more of the visual content, the webpage, a browser, or an operating system.

4. The method of claim 1 , wherein the visual content included in the generated test image of the webpage is determined to be visually consistent with the predetermined target image of the visual content when a number of pixels in the generated test image that match pixels in the predetermined target image is greater than a threshold number of pixels.

5. The method of claim 1 , wherein selectively providing the failure notification based upon the result of the comparison comprises:

providing the failure notification when the result of the comparison indicates that the visual content included in the generated test image of the webpage is not visually consistent with the predetermined target image of the visual content.

6. The method of claim 1 , wherein selectively providing the failure notification comprises:

notifying the user to review the generated test image of the webpage, and

the method further comprises:

receiving input specifying that the generated test image of the webpage is acceptable to the user; and

updating the predetermined target image of the visual content with a portion of the webpage that includes the visual content.

7. The method of claim 1 , further comprising:

generating the predetermined target image of the visual content based on live data received from a live system.

8. The method of claim 1 , wherein the predetermined target image of the visual content includes only the visual content without including other portions of the webpage.

9. The method of claim 1 , wherein comparing the generated test image with a predetermined target image of the visual content comprises:

comparing pixels of the generated test image with pixels of the predetermined target image.

10. A non-transitory computer storage medium encoded with instructions that when executed by a client device cause the client device to perform operations comprising:

receiving input specifying a change to a parameter relating to display of a visual content in a webpage;

in response to receiving the input, generating a test image of the webpage including the visual content, wherein the test image of the webpage represents a visual appearance of the webpage as displayed to a user of the client device;

comparing the generated test image with a predetermined target image of the visual content to determine whether the visual content included in the generated test image is visually consistent with the predetermined target image of the visual content, wherein the predetermined target image of the visual content represents an expected visual appearance of the visual content as displayed to the user of the client device, and the predetermined target image is stored in a repository; and

selectively providing a failure notification based upon a result of the comparison;

wherein comparing the generated test image with a predetermined target image of the visual content comprises:

selecting a plurality of different portions of the generated test image, where each portion has a same size as the predetermined target image,

determining, for each portion, a difference for the portion by comparing pixels of the portion with pixels of the predetermined target image,

identifying a portion of the plurality of different portions having a smallest difference, and

determining whether the smallest difference of the portion is smaller than a threshold difference;

wherein the visual content included in the generated test image of the webpage is determined to be visually consistent with the predetermined target image of the visual content when the smallest difference for the portion is smaller than the threshold difference.

11. The non-transitory computer storage medium of claim 10 , wherein the visual content comprises an advertisement.

12. The non-transitory computer storage medium of claim 10 , wherein receiving input specifying the change to the parameter comprises:

receiving input specifying a change to one or more of the visual content, the webpage, a browser, or an operating system.

13. The non-transitory computer storage medium of claim 10 , wherein the visual content included in the generated test image of the webpage is determined to be visually consistent with the predetermined target image of the visual content when a number of pixels in the generated test image that match pixels in the predetermined target image is greater than a threshold number of pixels.

14. The non-transitory computer storage medium of claim 10 , wherein selectively providing the failure notification based upon the result of the comparison comprises:

providing the failure notification when the result of the comparison indicates that the visual content included in the generated test image of the webpage is not visually consistent with the predetermined target image of the visual content.

15. The non-transitory computer storage medium of claim 10 , wherein selectively providing the failure notification comprises:

notifying the user to review the generated test image of the webpage, and

wherein the operations further comprise:

receiving input specifying that the generated test image of the webpage is acceptable to the user; and

updating the predetermined target image of the visual content with a portion of the webpage that includes the visual content.

16. The non-transitory computer storage medium of claim 10 , further comprising:

generating the predetermined target image of the visual content based on live data received from a live system.

17. The non-transitory computer storage medium of claim 10 , wherein the predetermined target image of the visual content includes only the visual content without including other portions of the webpage.

18. The non-transitory computer storage medium of claim 10 , wherein comparing the generated test image with a predetermined target image of the visual content comprises:

comparing pixels of the generated test image with pixels of the predetermined target image.

19. A system, comprising:

a data processing apparatus; and

a non-transitory computer storage medium encoded with instructions that when executed by a client device cause the client device to perform operations comprising:

receiving input specifying a change to a parameter relating to display of a visual content in a webpage;

in response to receiving the input, generating a test image of the webpage including the visual content, wherein the test image of the webpage represents a visual appearance of the webpage as displayed to a user of the client device;

comparing the generated test image with a predetermined target image of the visual content to determine whether the visual content included in the generated test image is visually consistent with the predetermined target image of the visual content, wherein the predetermined target image of the visual content represents an expected visual appearance of the visual content as displayed to the user of the client device, and the predetermined target image is stored in a repository; and

selectively providing a failure notification based upon a result of the comparison;

wherein comparing the generated test image with a predetermined target image of the visual content comprises:

selecting a plurality of different portions of the generated test image, wherein each portion has a same size as the predetermined target image;

determining, for each portion, a difference for the portion by comparing pixels of the portion with pixels of the predetermined target image;

identifying a portion of the plurality of different portions having a smallest difference; and

determining whether the smallest difference of the portion is smaller than a threshold difference;

wherein the visual content included in the generated test image of the webpage is determined to be visually consistent with the predetermined target image of the visual content when the smallest difference for the portion is smaller than the threshold difference.

20. The system of claim 19 , wherein the visual content comprises an advertisement.

21. The system of claim 19 , wherein receiving input specifying the change to the parameter comprises:

receiving input specifying a change to one or more of the visual content, the webpage, a browser, or an operating system.

22. The system of claim 19 , wherein the visual content included in the generated test image of the webpage is determined to be visually consistent with the predetermined target image of the visual content when a number of pixels in the generated test image that match pixels in the predetermined target image is greater than a threshold number of pixels.

23. The system of claim 19 , wherein selectively providing the failure notification based upon the result of the comparison comprises:

providing the failure notification when the result of the comparison indicates that the visual content included in the generated test image of the webpage is not visually consistent with the predetermined target image of the visual content.

24. The system of claim 19 , wherein selectively providing the failure notification comprises:

notifying the user to review the generated test image of the webpage, and

wherein the instructions that when executed by a client device cause the client device to perform operations further comprising:

receiving input specifying that the generated test image of the webpage is acceptable to the user; and

updating the predetermined target image of the visual content with a portion of the webpage that includes the visual content.

25. The system of claim 19 , further comprising:

generating the predetermined target image of the visual content based on live data received from a live system.

26. The system of claim 19 , wherein the predetermined target image of the visual content includes only the visual content without including other portions of the webpage.

27. The system of claim 19 , wherein comparing the generated test image with a predetermined target image of the visual content comprises:

comparing pixels of the generated test image with pixels of the predetermined target image.