Granted Patent B1

US 9,984,653 · App. 14/966,897 · Granted May 29, 2018

Abstract

Claims (51)

1. A method, comprising:

receiving compressed video data;

decoding the compressed video data into video frames and storing the video frames in a frame buffer for subsequent display by a display interface;

logging a decoding ending time and a display time for each of the video frames over a first window of time to generate a latency history including a sequence of records of the logged decoding ending times and display times, wherein the decoding ending time is a time at which the decoding of compressed video data for a corresponding video frame is completed and the display time is a time at which the display interface reads the corresponding video frame from the frame buffer; and

adjusting a display rate of the display interface based on the latency history over the first window of time in order to change waiting times of video frames such that an average waiting time of video frames is within a display latency window, each waiting time of a video frame being a difference between the decoding ending time of the video frame and the display time of the video frame.

2. The method of claim 1 , wherein adjusting the display rate of the display interface to change the waiting times of video frames based on the latency history includes:

determining an average waiting time of a first set of video frames based on the latency history over the first window of time; and

increasing the display rate of the display interface to reduce waiting times of a second set of video frames when the average waiting time of the first set of video frames is longer than a maximum display latency of the display latency window.

3. The method of claim 1 , wherein adjusting the display rate of the display interface to change the waiting times of video frames based on the latency history includes:

determining an average waiting time of a first set of video frames based on the latency history over the first window of time; and

decreasing the display rate of the display interface to increase waiting times of a second set of video frames when the average waiting time of the first set of video frames is shorter than a minimum display latency of the display latency window.

4. The method of claim 3 , wherein determining the average waiting time of the first set of video frames based on the latency history over the first window of time includes:

filtering the latency history to determine the average waiting time of the first set of video frames.

5. The method of claim 3 , wherein determining the average waiting time of the first set of video frames based on the latency history over the first window of time includes:

removing a momentarily changed waiting time from a sequence of waiting times used for determining the average waiting time of video frames, the sequence of waiting times being computed based on the latency history over the first window of time.

6. The method of claim 3 , wherein adjusting the display rate of the display interface to change the waiting times of video frames based on the latency history further includes:

determining a change rate of the display rate of the display interface for adjusting the display rate of the display interface, the change rate being in a range acceptable to a display device connected to the display interface.

7. The method of claim 3 , wherein adjusting the display rate of the display interface to change the waiting times of video frames based on the latency history further includes:

determining a change rate of the display rate of the display interface for adjusting the display rate of the display interface based on a trade-off between a convergence speed of the waiting times of video frames converging to the display latency window and a frame dropping rate caused by overshooting of adjustment of the display rate of the display interface.

8. The method of claim 1 , further comprising:

adjusting a maximum display latency or a minimum display latency of the display latency window according to a history of waiting times over a second window of time.

9. The method of claim 1 , further comprising:

synchronizing decoding starting times of video frames and the display times of the display interface based on timing information included in the compressed video data.

10. The method of claim 1 , further comprising:

after an adjustment of the display rate of the display interface, waiting for a predetermined period of time before performing another adjustment of the display rate of the display interface.

11. A device, comprising:

a decoder to receive compressed video data and decode the compressed video data into video frames;

a frame buffer to store the video frames;

a display interface to read the video frames from the frame buffer and display the video frames at a display rate;

a logger to log a decoding ending time and a display time for each of the video frames over a first window of time to generate a latency history including a sequence of records of the logged decoding ending times and display times, wherein the decoding ending time is a time at which the decoding of compressed video data for a corresponding video frame is completed and the display time is a time at which the display interface reads the corresponding video frame from the frame buffer; and

a display controller to adjust a display rate of the display interface based on the latency history over the first window of time in order to change waiting times of video frames such that an average waiting time of video frames is within a display latency window, each waiting time of a video frame being a difference between the decoding ending time of the video frame and the display time of the video frame.

12. The device of claim 11 , wherein the display controller is further configured to,

determine an average waiting time of a first set of video frames based on the latency history over the first window of time, and

increase the display rate of the display interface to reduce waiting times of a second set of video frames when the average waiting time of the first set of video frames is longer than a maximum display latency of the display latency window.

13. The device of claim 11 , wherein the display controller is further configured to,

determine an average waiting time of a first set of video frames based on the latency history over the first window of time; and

decrease the display rate of the display interface to increase waiting times of a second set of video frames when the average waiting time of the first set of video frames is shorter than a minimum display latency of the display latency window.

14. The device of claim 13 , wherein the display controller is configured to,

filter the latency history to determine the average waiting time of the first set of video frames.

15. The device of claim 13 , wherein the display controller is configured to,

remove a momentarily changed waiting time from a sequence of waiting times to determine the average waiting time of video frames, the sequence of waiting times being computed based on the latency history over the first window of time.

16. The device of claim 13 , wherein the display controller is configured to,

determine a change rate of the display rate of the display interface for adjusting the display rate of the display interface, the change rate being in a range acceptable to a display device connected to the display interface.

17. The device of claim 13 , wherein the display controller is configured to,

determine a change rate of the display rate of the display interface for adjusting the display rate of the display interface based on a trade-off between a convergence speed of the waiting times of video frames converging to the display latency window and a frame dropping rate caused by overshooting of adjustment of the display rate of the display interface.

18. The device of claim 11 , wherein the display controller is further configured to,

adjust a maximum display latency or a minimum display latency of the display latency window according to a history of waiting times over a second window of time.

19. The device of claim 11 , wherein the display controller is further configured to,

synchronize decoding starting times of video frames and the display times of the display interface based on timing information included in the compressed video data.

20. The device of claim 11 , wherein the display controller is further configured to,

after an adjustment of the display rate of the display interface, wait for a predetermined period of time before performing another adjustment of the display rate of the display interface.

Assignments (6)