Method and apparatus for automatic HLS bitrate adaptation
A media player system is provided for receiving and processing a media program that uses a time interval interval to required to decode N D frames of the media program segment. The media system receives the requested media program segment, processes the segment and determines if the throughput of the media program differs from the desired presentation throughput by a tolerance amount. Both decoding and rendering performance are determined and used to determine presentation throughput, and to determine if heavier or lighter variants of the media program should be requested for subsequent media program segments.
1. A method of receiving and processing a media program, the media program comprising a plurality of media program versions, each of the plurality of media program versions generated for a different presentation throughput than the other of the plurality of media program versions, each of the plurality of media program versions comprising a plurality of media program version segments, the method comprising:
transmitting a request for the media program;
receiving a master playlist for the requested media program, the master playlist comprising an index to plurality of media playlists, each media playlist having an address to each of a plurality of media program segments of a related variant of the media program suitable for a first presentation throughput, the first presentation throughput comprising a first communication throughput and a first processing throughput;
transmitting a request for a media program segment of the plurality of media program segments of a first variant of the media program;
receiving the requested media program segment;
processing the received media program segment;
determining a presentation throughput of the received media program segment, comprising:
determining a decoding performance of the received media program segment;
determining a rendering performance of the received media program segment; and
determining the presentation throughput at least in part from the determined decoding performance and the determined rendering performance;
determining if the presentation throughput of the received media program segment differs from a desired presentation throughput by more than a tolerance amount;
if the determined presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount, transmitting a request for a temporally following media program segment of another variant of the media program suitable for the determined presentation throughput; and
if the determined presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount, transmitting a request for a temporally following media program segment of the variant of the media program suitable for the determined presentation throughput; wherein
the step of determining the decoding performance of the received media program segment comprises determining a first time interval t D required to decode N D frames of the media program segment by: (i) determining a first time when the decoding of the N D frames is initiated according to a local clock; (ii) determining a second time when the decoding of the N D frames is completed according to the local clock; and (iii) determining the time interval N D according to a difference between the second time and the first time; and
the step of determining if the presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount comprises comparing at least the determined first time interval t D with a desired playback frame interval of the media program.
2. The method of claim 1 , wherein the desired playback frame interval is specified in the master playlist.
3. The method of claim 1 , wherein N D ≥2.
4. The method of claim 3 , wherein:
N
D
=
T
S
×
R
F
α
f
wherein T S is a temporal length of the media program segment, R F is a desired frame rate, and □ f is a dimensionless lead factor between two and three.
5. The method of claim 1 , wherein:
determining the rendering performance of the requested media program segment comprises:
measuring a second time interval t R required to render N R frames of the media program segment;
determining if the presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount comprises:
comparing a sum of the determined first time interval t D and the determined second time interval t R with the desired playback frame interval of the media program.
6. The method of claim 5 , wherein
determining the second time interval required to decode N R frames of the media program segment comprises:
determining a third time when the decoding of the N R frames is initiated according to a local clock;
determining a fourth time when the decoding of the N R frames is completed according to the local clock; and
determining the time interval t R according to a difference between the second time and the first time.
7. The method of claim 6 , wherein N R ≥2.
8. The method of claim 7 , wherein:
N
R
=
T
S
×
R
F
α
f
wherein T S is a temporal length of the media program segment, R F is a desired frame rate, and □ f is a dimensionless lead factor between two and three.
9. The method of claim 1 , wherein:
determining if the presentation throughput of the received media program segment is within the tolerance of the desired presentation throughput comprises:
determining a time interval between decoded frames provided from a decoder for rendering;
if the time interval between decoded frames provided for rendering exceeds a maximum time interval, determining that the presentation throughput differs from the desired presentation throughput by more than the tolerance amount.
10. The method of claim 1 , wherein:
at least one frame of the received media program segment comprises a presentation time stamp (PTS) and a decoding time stamp (DTS), and
determining if the presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount comprises:
decoding the at least one frame of the received media program segment;
determining a time which the at least one frame of the received media program segment is decoded;
rendering the decoded at least one frame of the received media program segment
determining a time which the at least one frame of the received media program segment is rendered;
determining that the presentation throughput of the received media program segment differs from the desired presentation throughput by more than the tolerance amount according to a comparison of:
the time which the at least one frame of the received media program segment is decoded and the DTS; and
the time which the at least one frame of the received media program segment is rendered and the PTS.
11. An apparatus for receiving and processing a media program, the media program comprising a plurality of media program versions, each of the plurality of media program versions generated for a different presentation throughput than the other of the plurality of media program versions, each of the plurality of media program versions comprising a plurality of media program version segments, the apparatus comprising:
a processor; and
a memory, communicatively coupled to the processor, the memory storing processor instructions comprising processor instructions for:
transmitting a request for the media program;
receiving a master playlist for the requested media program, the master playlist comprising an index to plurality of media playlists, each media playlist:
having an address to each of a plurality of media program segments of a related variant of the media program suitable for a first presentation throughput, the first presentation throughput comprising a first communication throughput and a first processing throughput;
transmitting a request for a media program segment of the plurality of media program segments of a first variant of the media program;
receiving the requested media program segment;
processing the received media program segment;
determining a presentation throughput of the received media program segment, comprising:
determining a decoding performance of the received media program segment;
determining a rendering performance of the received media program segment; and
determining the presentation throughput at least in part from the determined decoding performance and the determined rendering performance;
determining if the presentation throughput of the received media program segment differs from a desired presentation throughput by more than a tolerance amount;
if the determined presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount, transmitting a request for a temporally following media program segment of another variant of the media program suitable for the determined presentation throughput; and
if the determined presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount, transmitting a request for a temporally following media program segment of the variant of the media program suitable for the determined presentation throughput; wherein
the step of determining the decoding performance of the received media program segment comprises determining a first time interval t D required to decode N D frames of the media program segment by: (i) determining a first time when the decoding of the N D frames is initiated according to a local clock; (ii) determining a second time when the decoding of the N D frames is completed according to the local clock; and (iii) determining the time interval N D according to a difference between the second time and the first time; and
the step of determining if the presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount comprises comparing at least the determined first time interval t D with a desired playback frame interval of the media program.
12. The apparatus of claim 11 , wherein the desired playback frame interval is specified in the master playlist.
13. The apparatus of claim 11 , wherein:
N
D
=
T
S
×
R
F
α
f
wherein T S is a temporal length of the media program segment, R F is a desired frame rate, and □ f is a dimensionless lead factor between two and three.
14. The apparatus of claim 11 , wherein:
determining the rendering performance of the requested media program segment comprises:
determining a second time interval t R required to render N R frames of the media program segment, comprising:
determining a third time when the decoding of the N R frames is initiated according to a local clock;
determining a fourth time when the decoding of the N R frames is completed according to the local clock; and
determining the time interval t R according to a difference between the second time and the first time;
determining if the presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount comprises:
comparing a sum of the determined first time interval t D and the determined second time interval with the desired playback frame interval of the media program;
wherein:
N
R
=
T
S
×
R
F
α
f
,
wherein T S is a temporal length of the media program segment, R F is a desired frame rate, and □ f is a dimensionless lead factor between two and three.
15. The apparatus of claim 11 , wherein:
determining if the presentation throughput of the received media program segment differs from the desired presentation throughput by more than a tolerance amount comprises:
determining a time interval between decoded frames provided from a decoder for rendering;
if the time interval between decoded frames provided for rendering exceeds a threshold, determining that the presentation throughput differs from the desired presentation throughput by more than the tolerance amount.