IP Library Granted Patent US 9,674,518
Granted Patent B2
US 9,674,518 · App. 14/137,515 · Granted Jun 6, 2017

Measuring remote video display with embedded pixels

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,674,518
App. No.
14/137,515
Granted
Jun 6, 2017
Kind
B2
Abstract

Methods, systems, and computer programs are provided for managing remote display performance. One method includes an operation for determining pixel data for a group of macroblocks, each macroblock having a group of pixels. The macroblocks are for embedding in respective video frames of a video stream. The pixel data for each pixel in each macroblock is calculated with a formula based on the frame number of the respective video frame and on the location of the pixel within the macroblock. Farther, the method includes operations for embedding the macroblocks in the respective video frames, and for transmitting the video frames with the embedded macroblocks to a remote client. A performance metric for the transmitted video stream is calculated based on the macroblocks received at the remote client by comparing the received macroblocks to the expected macroblocks based on the formula.

Claims (79)

1. A method for determining a performance metric corresponding to a video stream transmitted from a server computing device to a remote client, comprising:

determining pixel data for a plurality of macroblocks, each macroblock having a plurality of pixels, the macroblocks defined to be embedded in respective video frames of the video stream, wherein the pixel data for each pixel in each macroblock is based on a frame number of a respective video frame and a location of the pixel within the macroblock; wherein determining pixel data further includes: calculating RGB values for each pixel, the RGB values including a red value, a green value, and a blue value;

embedding the macroblocks in the respective video frames;

transmitting the video frames with the embedded macroblocks to the remote client, wherein the video frames are displayed on the remote client;

wherein the remote client calculates expected macroblocks; and

wherein the performance metric for the transmitted video stream is calculated based on a comparison of the macroblocks received at the remote client and the expected macroblocks, wherein operations of the method are executed by a processor.

2. The method as recited in claim 1 , wherein calculating RGB values for each pixel further includes:

calculating the red value as ((k 1 ·n+k 2 ·x+k 3 ·y) modulo k 4 ) AND 0xFF;

calculating the green value as ((k 5 ·n+k 6 ·x+k 7 ·y) modulo k 8 ) AND 0xFF; and

calculating the blue value as ((k 9 ·n+k 10 ·x+k 11 ·y) modulo k 12 ) AND 0xFF,

wherein k 1 -k 12 are predetermined constants, n is the frame number, and x and y are coordinates of the pixel within the macroblock.

3. The method as recited in claim 2 , wherein a location of each macroblock within the respective video frame is different from a location of the macroblock inserted in a previous video frame.

4. A method for determining a performance metric corresponding to a video stream transmitted from a server computing device to a remote client, comprising:

determining pixel data for a plurality of macroblocks, each macroblock having a plurality of pixels, the macroblocks defined to be embedded in respective video frames of the video stream, wherein the pixel data for each pixel in each macroblock is based on a frame number of a respective video frame and a location of the pixel within the macroblock;

embedding the macroblocks in the respective video frames;

embedding metadata in pixel values of the macroblock;

transmitting the video frames with the embedded macroblocks to the remote client, wherein the video frames are displayed on the remote client;

calculating, at the remote client, expected macroblocks; and

wherein the performance metric for the transmitted video stream is calculated based on a comparison of the macroblocks received at the remote client and the expected macroblocks, wherein operations of the method are executed by a processor.

5. The method as recited in claim 4 , wherein the metadata includes a predetermined marker and the frame number.

6. The method as recited in claim 4 , wherein calculating the performance metric further includes:

determining first pixel values for a first expected macroblock based on the frame number of a received video frame and locations of each pixel within the first expected macroblock; and

comparing the first expected macroblock to a respective received macroblock embedded in the received video frame.

7. The method as recited in claim 4 , wherein calculating the performance metric further includes:

calculating the performance metric utilizing a Peak Signal-to-Noise Ratio (PSNR) algorithm based on comparing a received macroblock and an expected macroblock.

8. The method as recited in claim 4 , wherein embedding the macroblocks further includes:

embedding the macroblocks in the video stream; and

storing the video stream with the embedded macroblocks in permanent storage before transmitting the video frames.

9. A method for determining a performance metric corresponding to a video stream transmitted from a server computing device to a remote client, comprising:

determining pixel data for a plurality of macroblocks, each macroblock having a plurality of pixels, the macroblocks defined to be embedded in respective video frames of the video stream, wherein the pixel data for each pixel in each macroblock is based on a frame number of a respective video frame and a location of the pixel within the macroblock;

embedding the macroblocks in the respective video frames;

transmitting the video frames with the embedded macroblocks to the remote client, wherein the video frames are displayed on the remote client;

calculating, at the remote client, expected macroblocks; and

wherein the performance metric for the transmitted video stream is calculated based on a comparison of the macroblocks received at the remote client and the expected macroblocks, wherein operations of the method are executed by a processor;

wherein embedding the macroblocks further includes:

intercepting each video frame as the video frame is sent to the remote client; and

inserting the macroblock in each intercepted video frame before transmitting each video frame.

10. A non-transitory computer-readable storage medium storing a computer program for determining a performance metric corresponding to a video stream transmitted from a server computing device to a remote client, the computer-readable storage medium comprising:

program instructions for calculating pixel data for a plurality of macroblocks, each macroblock having a plurality of pixels, the macroblocks defined to be embedded in respective video frames of the video stream, wherein the pixel data for each pixel in each macroblock is calculated based on a frame number of the respective video frame and a location of the pixel within the macroblock, wherein calculating pixel data further includes:

program instructions for calculating RGB values for each pixel, the RGB values including a red value, a green value, and a blue value;

program instructions for embedding the macroblocks in the respective video frames; and

program instructions for transmitting the video frames with the embedded macroblocks to the remote client, wherein the video frames are displayed on the remote client, wherein the remote client is configured to:

calculate expected macroblocks;

calculate the performance metric for the transmitted video stream based on a comparison of the macroblocks received at the remote client and the expected macroblocks; and

store the performance metric in memory.

11. The non-transitory computer-readable storage medium as recited in claim 10 , wherein calculating RGB values further includes:

program instructions for calculating the red value as ((k 1 ·n+k 2 ·x+k 3 ·y) modulo k 4 ) AND 0xFF;

program instructions for calculating the green value as ((k 5 ·n+k 6 ·x+k 7 ·y) modulo k 8 ) AND 0xFF; and

program instructions for calculating the blue value a as ((k 9 ·n+k 10 ·x+k 11 ·y) modulo k 12 ) AND 0xFF, wherein k 1 -k 12 are predetermined constants, n is the frame number, and x and y are coordinates of the pixel within the macroblock.

12. The non-transitory computer-readable storage medium as recited in claim 10 , wherein embedding the macroblocks further includes:

program instructions for embedding the macroblocks in the video stream; and

program instructions for storing the video stream with the embedded macroblocks in permanent storage before transmitting the video frames.

13. A non-transitory computer-readable storage medium storing a computer program for determining a performance metric corresponding to a video stream transmitted from a server computing device to a remote client, the computer-readable storage medium comprising:

program instructions for calculating pixel data for a plurality of macroblocks, each macroblock having a plurality of pixels, the macroblocks defined to be embedded in respective video frames of the video stream, wherein the pixel data for each pixel in each macroblock is calculated based on a frame number of the respective video frame and a location of the pixel within the macroblock;

program instructions for embedding the macroblocks in the respective video frames; and

program instructions for transmitting the video frames with the embedded macroblocks to the remote client, wherein the video frames are displayed on the remote client, wherein the remote client is configured to:

calculate expected macroblocks;

calculate the performance metric for the transmitted video stream based on a comparison of the macroblocks received at the remote client and the expected macroblocks; and

store the performance metric in memory;

wherein embedding the macroblocks further includes:

program instructions for intercepting each video frame as the video frame is sent to the remote client; and

program instructions for inserting the macroblock in each intercepted video frame before transmitting each video frame.

14. A method for determining a performance metric corresponding to a video stream transmitted from a server computing device to a remote client computing device, comprising:

receiving, at the client device, a plurality of video frames of the video stream and displaying the video frames on the remote client, each video frame including a respective macroblock embedded in the video frame by the server, each macroblock having a plurality of pixels;

identifying received pixel data for pixels of each received macroblock;

determining expected pixel data for pixels of each received macroblock based on the frame number of the video frame and the location of the pixel within the macroblock; and

comparing the received pixel data to the expected pixel data for a plurality of pixels to calculate the performance metric for the received video stream, wherein operations of the method are executed by a processor and the performance metric is stored in memory;

wherein determining expected pixel data further includes:

calculating a red value as ((k 1 ·n+k 2 ·x+k 3 ·y) modulo k 4 ) AND 0xFF;

calculating a green value as ((k 5 ·n+k 6 ·x+k 7 ·y) modulo k 8 ) AND 0xFF; and

calculating a blue value a as ((k 9 ·n+k 10 ·x+k 11 ·y) modulo k 12 ) AND 0xFF,

wherein k 1 -k 12 are predetermined constants, n is the frame number, and x and y are coordinates of the pixel within the macroblock.

15. A method for determining a performance metric corresponding to a video stream transmitted from a server computing device to a remote client computing device, comprising:

receiving, at the client device, a plurality of video frames of the video stream and displaying the video frames on the remote client, each video frame including a respective macroblock embedded in the video frame by the server, each macroblock having a plurality of pixels;

identifying received pixel data for pixels of each received macroblock;

determining expected pixel data for pixels of each received macroblock based on the frame number of the video frame and the location of the pixel within the macroblock; and

comparing the received pixel data to the expected pixel data for a plurality of pixels to calculate the performance metric for the received video stream, wherein operations of the method are executed by a processor and the performance metric is stored in memory;

wherein the macroblocks are embedded at the server by intercepting each video frame as the video frame is sent to the remote client, and inserting the macroblock before transmitting each video frame.

16. The method as recited in claim 14 , wherein operations of the method are performed by a computer program when executed by one or more processors, the computer program being embedded in a non-transitory computer-readable storage medium.

Assignments (4)
PATENT ASSIGNMENT Recorded Aug 5, 2024
From: VMWARE LLC
To: OMNISSA, LLC
Reel/Frame 068327/0365 →
SECURITY INTEREST Recorded Jul 3, 2024
From: OMNISSA, LLC
To: UBS AG, STAMFORD BRANCH
Reel/Frame 068118/0004 →
CHANGE OF NAME Recorded Apr 15, 2024
From: VMWARE, INC.
To: VMWARE LLC
Reel/Frame 067102/0314 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2013
From: AGRAWAL, BANIT; SPRACKLEN, LAWRENCE; BIDARKAR, RISHI
To: VMWARE, INC.
Reel/Frame 031834/0341 →