Perceptually-aware online per-title encoding for live video streaming
Techniques for implementing perceptually aware per-title encoding may include receiving an input video, a set of resolutions, a maximum target bitrate and a minimum target bitrate, extracting content aware features for each segment of the input video, predicting a perceptually aware bitrate-resolution pair for each segment using a model configured to optimize for a quality metric using constants trained for each of the set of resolutions, generating a target encoding set including a set of perceptually aware bitrate-resolution pairs, and encoding the target encoding set. The content aware features may include a spatial energy feature and an average temporal energy. According to these methods only a subset of bitrates and resolutions, less than a full set of bitrates and resolutions, are encoded to provide high quality video content for streaming.
1. A method for perceptually aware per-title encoding, the method comprising:
receiving an input video, a set of resolutions, a maximum target bitrate and a minimum target bitrate;
extracting two or more features for each of a plurality of segments of the input video, the two or more features comprising a spatial energy feature and an average temporal energy;
predicting a perceptually aware bitrate-resolution pair for the each of the plurality of segments using a model, the perceptually aware bitrate-resolution pair being based on the spatial energy feature and the average temporal energy, the model being configured to optimize for a quality metric using constants trained for each resolution in the set of resolutions;
generating a target encoding set comprising a set of perceptually aware bitrate-resolution pairs; and
encoding the target encoding set.
2. The method of claim 1 , wherein the set of perceptually aware bitrate-resolution pairs comprises fewer pairs than a number of resolutions in the set of resolutions.
3. The method of claim 1 , wherein the set of perceptually aware bitrate-resolution pairs comprises fewer pairs than a number of bitrates from the maximum target bitrate to the minimum target bitrate.
4. The method of claim 1 , wherein the perceptually aware bitrate-resolution pair is predicted based on a just noticeable difference (JND) quality value.
5. The method of claim 1 , wherein the two or more features are extracted using a Discrete Cosine Transform (DCT)-energy function.
6. The method of claim 5 , wherein exponentially higher costs are assigned to higher DCT frequencies.
7. The method of claim 1 , wherein the two or more features comprise low-complexity features.
8. The method of claim 1 , wherein the average temporal energy is computed by comparing the blockwise sum of absolute differences (SAD) of the texture energy of each frame in a given segment of the plurality of segments to the blockwise SAD of a previous frame in the given segment.
9. The method of claim 1 , wherein the quality metric comprises video multi-method assessment fusion (VMAF).
10. The method of claim 1 , wherein the constants trained for each resolution are trained using a linear regression approach.
11. The method of claim 1 , wherein predicting the perceptually aware bitrate-resolution pair comprises implementing a machine learning model.
12. The method of claim 1 , wherein the encoding the target encoding set comprises online per-title encoding.
13. The method of claim 1 , further comprising receiving as input one, or a combination, of a peak bitrate, a maximum buffer size, and a maximum segment length.
14. The method of claim 1 , further comprising scaling the encoded target encoding set for display.