IP Library Granted Patent US 11,930,299
Granted Patent B2
US 11,930,299 · App. 17/216,829 · Granted Mar 12, 2024

Measuring audio and video latencies in virtual desktop environments

Inventors: Srinivas Shyam Pinjala (Bangalore, IN); Lakshmi Gayatri Kundem (Bangalore, IN); Duraipandian Kuppuraman (Bangalore, IN); Janani Karthikeyan (Tiruppur, IN)
Assignee: VMware LLC
H04N7/157G06F9/45558H04N7/147G06F2009/4557G06F2009/45579G06F2009/45595
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,930,299
App. No.
17/216,829
Granted
Mar 12, 2024
Kind
B2
Abstract

Described herein are systems, methods, and software to monitor latency information in virtual desktop environments. In one example, a user computing system may obtain a first frame of video data from a second computing system, wherein the video data is streamed from the user computing system to the second computing system. The user computing system further identifies a first frame number for the first frame based on a code in the first frame, identifies a frame number for second frame of the video data to be streamed to the second computing system when first frame was received, and determines frame latency based on a difference between the first frame number and the second frame number.

Claims (56)

1. A method of operating a first computing system, the method comprising:

streaming video data from the first computing system to a virtual desktop executing on a second computing system, wherein the virtual desktop executes an application to view the video data;

receiving a first frame of the video data as part of a display of the virtual desktop from the second computing system;

identifying a first frame number for the first frame based on a code embedded in the first frame;

identifying a second frame of the video data to be streamed by the first computing system to the virtual desktop when the first frame was received;

identifying a second frame number associated with the second frame; and

determining frame latency based on a difference between the first frame number and the second frame number.

2. The method of claim 1 , wherein the second frame of the video data comprises a second code indicating at least the second frame number, and wherein the method further comprises:

communicating the second frame to the virtual desktop executing on the second computing system.

3. The method of claim 1 , wherein the application comprises a video conferencing application or live video application.

4. The method of claim 1 further comprising:

receiving a third frame of the video data from the second computing system;

identifying expected audio data for the third frame based on a code embedded in the third frame;

monitoring output audio data for the first computing system; and

determining audio latency based on a difference between the output audio data and the expected audio data.

5. The method of claim 4 further comprising:

receiving one or more additional frames of the video data from the second computing system;

identifying expected audio data for the one or more additional frames of the video data based on codes embedded in each of the one or more additional frames;

determining the audio latency as a function of time based on a comparison of the output audio data and the expected audio data.

6. The method of claim 1 , wherein the code comprises a Quick Response (QR) code.

7. The method of claim 1 further comprising:

generating a display indicative of the frame latency.

8. The method of claim 7 , wherein the display indicates a quality of service based on the frame latency.

9. A computing apparatus comprising:

a storage system;

a processing system operatively coupled to the storage system;

program instructions stored on the storage system that, when executed by the processing system, direct the computing apparatus to:

stream video data from the first computing system to a virtual desktop executing on a second computing system, wherein the virtual desktop executes an application to view the video data;

receive a first frame of the video data as part of a display of the virtual desktop from the second computing system;

identify a first frame number for the first frame based on a code embedded in the first frame;

identify a second frame of the video data to be streamed by the first computing system to the virtual desktop when the first frame was received;

identify a second frame number associated with the second frame; and

determine frame latency based on a difference between the first frame number and the second frame number.

10. The computing apparatus of claim 9 , wherein the second frame of the video data comprises a second code indicating at least the second frame number, and wherein the program instructions further direct the computing apparatus to:

communicate the second frame to the virtual desktop executing on the second computing system.

11. The computing apparatus of claim 9 , wherein the application comprises a video conferencing application or live video application.

12. The computing apparatus of claim 9 , wherein the program instructions further direct the computing apparatus to:

receive a third frame of the video data from the second computing system;

identify expected audio data for the third frame based on a code embedded in the third frame;

monitor output audio data for the first computing system; and

determine audio latency based on a difference between the output audio data and the expected audio data.

13. The computing apparatus of claim 12 , wherein the program instructions further direct the computing apparatus to:

receive one or more additional frames of the video data from the second computing system;

identify expected audio data for the one or more additional frames of the video data based on codes embedded in each of the one or more additional frames;

determine the audio latency as a function of time based on a comparison of the output audio data and the expected audio data.

14. The computing apparatus of claim 9 , wherein the code comprises a Quick Response (QR) code.

15. The computing apparatus of claim 9 , wherein the program instructions further direct the computing apparatus to generate a display indicative of the frame latency.

16. The computing apparatus of claim 15 , wherein the display indicates a quality of service associated with the frame latency.

17. A method of operating a first computing system comprising:

receiving one or more frames of video data from a second computing system;

identifying expected audio data for the one or more frames based on a code embedded in each of the one or more frames;

monitoring output audio data at the first computing system while displaying the one or more frames, wherein the output audio data is received at least partially from the second computing system;

determining a synchronization status for the output audio data and the one or more frames based on a comparison of the expected audio data and the output audio data.

18. The method of claim 17 , wherein the code comprises a Quick Response (QR) code.

19. The method of claim 17 further comprising generating a display of the synchronization status.

20. The method of claim 17 , wherein the synchronization status comprises latency associated with the output audio data.

Assignments (4)
PATENT ASSIGNMENT Recorded Aug 5, 2024
From: VMWARE LLC
To: OMNISSA, LLC
Reel/Frame 068327/0365 →
SECURITY INTEREST Recorded Jul 3, 2024
From: OMNISSA, LLC
To: UBS AG, STAMFORD BRANCH
Reel/Frame 068118/0004 →
CHANGE OF NAME Recorded Feb 27, 2024
From: VMWARE, INC.
To: VMWARE LLC
Reel/Frame 066692/0103 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 30, 2021
From: PINJALA, SRINIVAS SHYAM; KUNDEM, LAKSHMI GAYATRI; KUPPURAMAN, DURAIPANDIAN; KARTHIKEYAN, JANANI
To: VMWARE, INC.
Reel/Frame 055763/0249 →
Priority Claims (1)
IN 202141003129 · Jan 22, 2021 · national
Continuity (1)
Related Publication 20220239865A1 · Jul 28, 2022
Cited By (1)
US 12,211,522