Systems and methods of video jitter estimation
A method and apparatus of a device that uses a video jitter buffer to buffer the video frames for a received video stream is described. The device uses the video jitter buffer to estimate the delay variability of the frames and pick a target delay that will help harmonize the playback with minimal loss and delay. This is achieved by estimating the delay spread, which is the result of performing statistical analysis of the measured lags of the video frames received. The video jitter buffer provides target delay recommendation and reference frame information to the video player in order to anchor the playback and schedule the frames presentation time.
1. A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method to determine a frame playback scheduling for a video stream, the method comprising:
receiving, on a device, the video stream, wherein the video stream includes a plurality of video frames;
determining a reference frame that is used to measure a target delay for the video stream, wherein the reference frame is a frame that is received after a jitter buffer fills up;
determining the target delay for the video stream, wherein the device uses the reference frame and the target delay to configure frame playback scheduling for the device by, computing a target delay using a percentile of a statistical distribution of multiple frame lag times between the reference frame and a subset of the plurality of video frames, wherein the statistical distribution includes a short time frame lag time distribution and a long time frame lag time distribution.
2. The non-transitory machine-readable medium of claim 1 , wherein the target delay is estimated using a statistical analysis of a plurality of adjusted lag times computed from the plurality of video frames.
3. The non-transitory machine-readable medium of claim 1 , further comprising:
computing a plurality of frame lag times for each of the plurality of video frames.
4. The non-transitory machine-readable medium of claim 3 , wherein the determining of the reference frame is based on at least the plurality of frame lag times.
5. The non-transitory machine-readable medium of claim 3 , wherein the determining of the reference frame further comprises:
computing a statistically meaningful minimum lag by using at least a kurtosis coefficient and a skewness coefficient when one of the plurality of frame lag times is less than a current minimum lag.
6. The non-transitory machine-readable medium of claim 3 , wherein a lag time from the plurality of frame lag times is statistically relevant when that lag time is greater than a minimum lag threshold.
7. The non-transitory machine-readable medium of claim 3 , further comprises:
adjusting the plurality of frame lag time to generate a plurality of adjusted lag times.
8. The non-transitory machine-readable medium of claim 7 , further comprising:
assigning each of the plurality of adjusted lag times to a bin based on a value of that adjusted lag time.
9. The non-transitory machine-readable medium of claim 8 , further comprising:
determining a cumulative distribution function based on the distribution of adjusted lag times.
10. The non-transitory machine-readable medium of claim 9 , wherein the determining of the target delay is from at least the cumulative distribution function.
11. The non-transitory machine-readable medium of claim 1 , wherein the video stream is associated with a mode based on at least a use of the video stream and the mode is used to dynamically adjust jitter buffer behavior for a video jitter buffer.
12. The non-transitory machine-readable medium of claim 11 , wherein the mode can be one of latency sensitive and erasure sensitive.
13. The non-transitory machine-readable medium of claim 11 , wherein the mode of the video stream is based on at least a content type of the video stream.
14. The non-transitory machine-readable medium of claim 1 , wherein at least one of the target delay and reference frame can change based on at least different lag times of the plurality of video frames.
15. The non-transitory machine-readable medium of claim 1 , wherein the device uses the reference frame to anchor frame playback.
16. The non-transitory machine-readable medium of claim 1 , wherein the short time frame lag time distribution has a shorter time window that the long time frame lag time distribution.
17. The non-transitory machine-readable medium of claim 1 , wherein a frame lag time distribution is a cumulative distribution function.
18. A method to determine a frame playback scheduling for a video stream, the method comprising:
receiving, on a device, the video stream, wherein the video stream includes a plurality of video frames;
determining a reference frame that is used to measure a target delay for the video stream, wherein the reference frame is a frame that is received after a jitter buffer fills up;
determining the target delay for the video stream, wherein the device uses the reference frame and the target delay to configure frame playback scheduling for the device by,
computing a target delay using a percentile of a statistical distribution of multiple frame lag times between the reference frame and a subset of the plurality of video frames, wherein the statistical distribution includes a short time frame lag time distribution and a long time frame lag time distribution.
19. The method of claim 18 , further comprising:
computing a plurality of frame lag times for each of the plurality of video frames.
20. The method of claim 19 , wherein the determining of the reference frame is based on at least the plurality of frame lag times.
21. The method of claim 20 , wherein the determining of the reference frame further comprises:
computing a statistically meaningful minimum lag by using at least a kurtosis coefficient and a skewness coefficient when one of the plurality of frame lag times is less than a current minimum lag.
22. The method of claim 18 , wherein the short time frame lag time distribution has a shorter time window that the long frame time lag time distribution.
23. The method of claim 18 , wherein a frame lag time distribution is a cumulative distribution function.