Quantitative visual perception quality measurement for virtual desktops
View Patent ↗Techniques are described for improving the measurement of visual perception of graphical user interface (GUI) information remoted to client devices in virtual desktop environments, such as VDI and DAAS. An objective image quality measurement of remoted virtual desktop interfaces is computed, that is more accurate and more closely aligned with subjective user perception. The visual quality metric is computed using a linear fusion model that combines a peak signal to noise ratio (PSNR) score of the distorted image, a structural similarity (SSIM) score of the distorted image and a feature similarity (FSIM) score of the distorted image. Prior to using the model to compute the quantitative visual perception metric, the linear fusion model is trained by using a benchmark test database of reference images (e.g., virtual desktop interface images), distorted versions of those images and subjective human visual perception quality ratings associated with each distorted version.
1. A method for quantitatively measuring visual perception of display information of a virtual desktop, the method comprising:
obtaining a set of reference virtual desktop images including one or more of desktop background images, text, icons, windows, or menus;
generating distorted versions of the reference virtual desktop images that have been distorted by processes involved in remoting a graphical user interface of the virtual desktop;
receiving a set of subjective visual quality ratings corresponding to the distorted versions of the reference virtual desktop images;
training a linear fusion model using as input the set of reference virtual desktop images, the distorted versions and the set of subjective visual quality ratings to determine a set of weightings, wherein the linear fusion model combines a peak signal to noise ratio (PSNR) score, a structural similarity (SSIM) score and a feature similarity (FSIM) score according to the determined weightings;
receiving a distorted image of a virtual desktop interface displayed by a remote server, the distorted image received by a client device from the remote server over a network connection;
displaying the distorted image of the virtual desktop interface by the client device and capturing a screenshot of the distorted image;
comparing the screenshot of the distorted image to a distortion-free reference image by the client device; and
computing a visual quality metric for the screenshot of the distorted image using the trained linear fusion model, wherein resource allocation for hosting the virtual desktop is adjusted based at least in part on the visual quality metric.
2. The method of claim 1 , further comprising:
uploading the visual quality metric of the distorted image to a central planner; and
processing a plurality of visual quality metrics received from a plurality of client devices by the central planner to compute an average visual quality metric of the distorted image among the plurality of client devices.
3. The method of claim 1 , wherein the linear fusion model combines the PSNR score, SSIM score and FSIM score according to a relationship:
visual quality metric=−18.17+0.02(PSNR)+3.44(SSIM)+17.83(FSIM).
4. The method of claim 1 , wherein the visual quality metric is used to benchmark one or more of: different CPU architectures, different host computing devices, different storage architectures, different configurations or different optimizations.
5. The method of claim 4 , wherein the visual quality metric is combined with a metric measuring responsiveness of the virtual desktop interface at the client device.
6. The method of claim 1 , wherein the virtual desktop is executed on a virtual machine managed by a hypervisor executed on the server, the virtual machine associated with a user of the first client device.
7. A non-transitory computer readable storage medium comprising one or more sequences of instructions, the instructions when executed by one or more processors causing the one or more processors to execute the operations of:
obtaining a set of reference virtual desktop images including one or more of desktop background images, text, icons, windows, or menus;
generating distorted versions of the reference virtual desktop images that have been distorted by processes involved in remoting a graphical user interface of the virtual desktop;
receiving a set of subjective visual quality ratings corresponding to the distorted versions of the reference virtual desktop images;
training a linear fusion model using as input the set of reference virtual desktop images, the distorted versions and the set of subjective visual quality ratings to determine a set of weightings, wherein the linear fusion model combines a peak signal to noise ratio (PSNR) score, a structural similarity (SSIM) score and a feature similarity (FSIM) score according to the determined weightings;
receiving a distorted image of a virtual desktop interface displayed by a remote server, the distorted image received by a client device from the remote server over a network connection;
displaying the distorted image of the virtual desktop interface by the client device and capturing a screenshot of the distorted image;
comparing the screenshot of the distorted image to a distortion-free reference image by the client device; and
computing a visual quality metric for the screenshot of the distorted image using the trained linear fusion model, wherein resource allocation for hosting the virtual desktop is adjusted based at least in part on the visual quality metric.
8. The non-transitory computer readable storage medium of claim 7 , further comprising instructions to cause the one or more processors to execute the operations of:
uploading the visual quality metric of the distorted image to a central planner; and
processing a plurality of visual quality metrics received from a plurality of client devices by the central planner to compute an average visual quality metric of the distorted image among the plurality of client devices.
9. The non-transitory computer readable storage medium of claim 7 , wherein the linear fusion model combines the PSNR score, SSIM score and FSIM score according to a relationship:
visual quality metric=−18.17+0.02(PSNR)+3.44(SSIM)+17.83(FSIM).
10. The non-transitory computer readable storage medium of claim 7 , wherein the visual quality metric is used to benchmark one or more of: different CPU architectures, different host computing devices, different storage architectures, different configurations or different optimizations.
11. The non-transitory computer readable storage medium of claim 10 , wherein the visual quality metric is combined with a metric measuring responsiveness of the virtual desktop interface at the client device.
12. The non-transitory computer readable storage medium of claim 7 , wherein the virtual desktop is executed on a virtual machine managed by a hypervisor executed on the server, the virtual machine associated with a user of the first client device.
13. A computing device, comprising:
one or more processors; and
memory including instructions that when executed by the one or more processors, cause the computing device to:
obtain a set of reference virtual desktop images including one or more of desktop background images, text, icons, windows, or menus;
generate distorted versions of the reference virtual desktop images that have been distorted by processes involved in remoting a graphical user interface of the virtual desktop;
receive a set of subjective visual quality ratings corresponding to the distorted versions of the reference virtual desktop images;
train a linear fusion model using as input the set of reference virtual desktop images, the distorted versions and the set of subjective visual quality ratings to determine a set of weightings, wherein the linear fusion model combines a peak signal to noise ratio (PSNR) score, a structural similarity (SSIM) score and a feature similarity (FSIM) score according to the determined weightings;
receive a distorted image of a virtual desktop interface displayed by a remote server, the distorted image received by a client device from the remote server over a network connection;
display the distorted image of the virtual desktop interface by the client device and capturing a screenshot of the distorted image;
compare the screenshot of the distorted image to a distortion-free reference image by the client device; and
compute a visual quality metric for the screenshot of the distorted image using linear fusion model, wherein resource allocation for hosting the virtual desktop is adjusted based at least in part on the visual quality metric.
14. The computing device of claim 13 , wherein the memory further includes instructions that cause the computing device to:
uploading the visual quality metric of the distorted image to a central planner; and
processing a plurality of visual quality metrics received from a plurality of client devices by the central planner to compute an average visual quality metric of the distorted image among the plurality of client devices.
15. The computing device of claim 13 , wherein the linear fusion model combines the PSNR score, SSIM score and FSIM score according to a relationship:
visual quality metric=−18.17+0.02(PSNR)+3.44(SSIM)+17.83(FSIM).
16. The computing device of claim 13 , wherein the visual quality metric is used to benchmark one or more of: different CPU architectures, different host computing devices, different storage architectures, different configurations or different optimizations.
17. The computing device of claim 16 , wherein the visual quality metric is combined with a metric measuring responsiveness of the virtual desktop interface at the client device.