Method for shortening display latency based on variable refresh rate technology and related rendering device thereof

1 . A method for displaying a video stream on a display device with a variable refresh rate function, comprising:

receiving the video stream and decoding it to generate a plurality of decoded video frames;

determining a frame ready time for each of the decoded video frames, wherein the frame ready time corresponds to a time that a decoded video frame is written into a storage unit;

determining a frame display time for each of the decoded video frames, wherein the frame display time corresponds to an expected display time of a decoded video frame;

determining a target refresh rate for the display device based on a frame latency between the frame ready time and the frame display time corresponding to at least one of the decoded video frames, wherein the target refresh rate is selected from one of a predetermined and constant low refresh rate that is lower than a frame rate of the video stream and a predetermined and constant high refresh rate that is higher than the frame rate of the video stream, comprising:

determining an average frame latency based on frame latencies corresponding to the plurality of the decoded video frames;

selecting a refresh rate that is lower than the frame rate as the target refresh rate when the average frame latency falls within a first range between 0 and a predetermined video latency threshold; and

selecting a refresh rate that is higher than the frame rate as the target refresh rate when the average frame latency falls within a second range between the predetermined video latency threshold and a frame period of the plurality of the decoded video frames; and

controlling the display device to display the video stream at the target refresh rate.

2 . The method of claim 1 , further comprising:

controlling the display device to operate at the target refresh rate when the frame latency exceeds a predetermined value; and

controlling the display device to operate at a default refresh rate when the frame latency is below the predetermined value.

3 . The method of claim 2 , wherein the step of determining the target refresh rate for the display device comprises:

setting a refresh rate that is lower than the default refresh rate as the target refresh rate if the default refresh rate of the display device corresponds to a frame rate of the video stream and the frame latency exceeds the predetermined value.

4 . The method of claim 2 , wherein the step of determining the target refresh rate for the display device comprises:

setting a refresh rate that is lower than the default refresh rate as the target refresh rate if the frame latency falls within a first numerical range; and

setting a refresh rate that is higher than the default refresh rate as the target refresh rate if the frame latency falls within a second numerical range.

5 . The method of claim 1 , wherein the step of determining the target refresh rate for the display device comprises:

determining an average frame latency based on frame latencies between the frame ready time and the frame display time of a specific number of decoded video frames among the plurality of decoded video frames; and

determining the target refresh rate based on the average frame latency, a target frame latency, and a variable refresh rate control period.

6 . The method of claim 1 , wherein the video stream is provided by a video source, and the video source is a cloud gaming service platform or a wireless projection source device.

7 . The method of claim 1 , wherein the frame ready time of a decoded video frame corresponds to the time when the decoded video frame is written to a storage unit of the rendering device, and the frame display time of the decoded video frame corresponds to the timing of a vertical synchronization signal corresponding to displaying the decoded video frame on the display device.

8 . A rendering device for controlling a display device with a variable refresh rate function, comprising:

a decoding unit configured to receive a video stream and decoding the video stream to generate a plurality of decoded video frames; and

a refresh rate control unit, coupled to the decoding unit, configured to determine a frame ready time for each of the decoded video frames, and a frame display time for each of the decoded video frames, and accordingly determine a target refresh rate for the display device based on a frame latency between the frame ready time and the frame display time corresponding to at least one of the decoded video frames, wherein the frame ready time corresponds to a time that a decoded video frame is written into a storage unit and the frame display time corresponds to an expected display time of a decoded video frame; the target refresh rate is selected from one of a predetermined and constant low refresh rate that is lower than a frame rate of the video stream and a predetermined and constant high refresh rate that is higher than the frame rate of the video stream;

wherein the refresh rate control unit is further configured to:

determine an average frame latency based on frame latencies corresponding to the plurality of the decoded video frames;

select a refresh rate that is lower than the frame rate as the target refresh rate when the average frame latency falls within a first range between 0 and a predetermined video latency threshold; and

select a refresh rate that is higher than the frame rate as the target refresh rate when the average frame latency falls within a second range between the predetermined video latency threshold and a frame period of the plurality of the decoded video frames; and

wherein the rendering device is configured to control the display device to display the video stream at the target refresh rate.

9 . The rendering device of claim 8 , wherein the rendering device is configured to:

control the display device to operate at the target refresh rate if an average frame latency exceeds a predetermined value; and

control the display device to operate at a default refresh rate if the average frame latency is below the predetermined value.

10 . The rendering device of claim 9 , wherein the refresh rate control unit is configured to set a refresh rate that is below the default refresh rate as the target refresh rate if the default refresh rate of the display device corresponds to a frame rate of the video stream and the frame latency exceeds the predetermined value.

11 . The rendering device of claim 9 , wherein the refresh rate control unit is configured to:

set a refresh rate that is lower than the default refresh rate as the target refresh rate if the frame latency falls within a first numerical range; and

set a refresh rate that is higher than the default refresh rate as the target refresh rate if the frame latency falls within a second numerical range.

12 . The rendering device of claim 8 , wherein the refresh rate control unit is configured to:

determine an average frame latency based on frame latencies between the frame ready time and the frame display time of a specific number of decoded video frames among the plurality of decoded video frames; and

determine the target refresh rate based on the average frame latency, a target frame latency, and a variable refresh rate control period.

13 . The rendering device of claim 8 , the video stream is provided by a video source, and the video source is a cloud gaming service platform or a wireless projection source device.

14 . The rendering device of claim 8 , wherein the frame ready time of a decoded video frame corresponds to the time when the decoded video frame is written to a storage unit of the rendering device, and the frame display time of the decoded video frame corresponds to the timing of a vertical synchronization signal corresponding to displaying the decoded video frame on the display device.