Video asset quality assessment and encoding optimization to achieve target quality requirement
Aspects of the present disclosure relate to the assessment on how well and consistent the quality of a video asset satisfies a given target quality level or a time-varying target quality curve, and the optimization on encoding configuration to achieve the best compromise between satisfying the target quality requirement and saving the bit rate/bandwidth cost. The application scope of the present disclosure is generally in, but not limited to, the field of video coding and distributions, including both live and file-based video encoding, broadcasting and streaming systems. Methods and systems implemented based on the present disclosure may achieve the highest accuracy approaching any given target quality with the smoothest quality variation over time, while maximally reduce bit rate/bandwidth and video distribution cost by optimally determining video encoding configurations and parameters.
1 . A method for assessing video quality of a video asset, comprising:
inputting the video asset, wherein the video asset includes a sequence of image frames with a plurality of time instances;
assessing the video quality of the video asset at the plurality of time instances of the video asset to determine a plurality of raw per-instance quality measures;
inputting a time-varying target quality;
using the time-varying target quality to adjust the raw per-instance quality measures to produce a plurality of adjusted per-instance quality measures; and
determining an overall quality score of the video asset based on the plurality of the adjusted per-instance quality measures wherein adjusting per-instance quality measures includes clipping the raw per-instance quality measures to a cap value that is computed as a function of the time-varying target quality.
2 . The method of claim 1 , further comprising:
creating a time-varying target quality curve based on user input with respect to the time-varying target quality at each of the plurality of time instances.
3 . The method of claim 1 , further comprising, in assessing the video quality at each time instance of the plurality of time instances, one or more of:
assessing the video quality at a frame of the video asset corresponding to the respective time instance, or
assessing the video quality of a time segment composed of multiple consecutive frames of the video asset near the respective time instance.
4 . The method of claim 3 , wherein the assessing of the video quality at each of the plurality of time instances further includes conducting subjective testing.
5 . The method of claim 3 , wherein the assessing of the video quality at each of the plurality of time instances further includes computing an objective video quality metric.
6 . The method of claim 3 , wherein the assessing of the video quality at each of the plurality of time instances further includes inferring the video quality as a monotonically increasing function with respect to a video bit rate or a resolution of the video asset.
7 . The method of claim 1 , further comprising, in computing the cap value as the function of the time-varying target quality, defining the function as the time-varying target quality plus a delta value that is either constant or monotonically increasing with the time-varying target quality value.
8 . The method of claim 1 , further comprising, in aggregating the adjusted per-instance quality measures into an overall quality assessment of the video asset, computing one or more of an average, percentile average, weighted average, or median of the adjusted per-instance quality measures.
9 . The method of claim 1 , further comprising, in aggregating the adjusted per-instance quality measures into an overall quality assessment of the video asset:
extracting a time window from the video asset;
computing a quality assessment of the time window as an average, a percentile average, a weighted average, or a median of the adjusted per-instance quality measures over the time window;
moving starting and ending points of the time window over time; and
creating a time varying quality curve by the quality assessment of the time window as moved.
10 . The method of claim 1 , further comprising:
encoding the video asset using a coding configuration that maximizes overall quality of the video asset.
11 . The method of claim 10 , further comprising:
segmenting the video asset into time windows;
encoding the video asset within each time window of the time windows using the coding configuration that satisfies target quality requirements of the plurality of time instances within the time window and minimizes a bit rate of the time window; and
connecting the encoding of the time windows into a full encoding of the video asset.
12 . The method of claim 10 , further comprising encoding the video asset by:
segmenting the video asset into time windows;
encoding the video asset within each time window of the time windows using the coding configuration that maximizes a joint optimization objective function determined based on an average bit rate and a quality assessment of the time window under the time-varying target quality; and
connecting the encoding of the time windows into a full encoding of the video asset.
13 . The method of claim 12 , further comprising computing the joint optimization objective function as a sum of:
the overall quality of a video asset or the quality assessment of a time window; and
a product of the average bit rate of the video asset and a Lagrange parameter.
14 . The method of claim 10 , further comprising defining the coding configuration as a collection of a coding parameter P assigned to each of the time instances, the coding parameter P including one or more of a quantization parameter, a quality control parameter, or a bit rate.
15 . The method of claim 14 , further comprising:
computing a derivative of the overall quality of the video asset with respect to the coding parameter P;
computing a derivative of an average bit rate of the video asset with respect to the coding parameter P;
determining an optimal Lagrange parameter based on a ratio between the derivative of the overall quality of the video asset with respect to the coding parameter P and the derivative of the average bit rate of the video asset with respect to the coding parameter P; and
determining the coding parameter P using the optimal Lagrange parameter.
16 . The method of claim 1 , wherein the cap value specifies a constant maximum value for the raw per-instance quality measures below a maximum possible quality score, such that any raw per-instance quality measures exceeding the maximum possible quality score are reduced to the constant maximum value in computing the overall quality score of the video asset.
17 . A system for assessing video quality of a video asset, comprising:
a computing device programmed to:
input the video asset, wherein the video asset includes a sequence of image frames with a plurality of time instances;
assess the video quality of the video asset at the plurality of time instances to determine a plurality of raw per-instance quality measures;
input a time-varying target quality;
use the time-varying target quality to adjust the raw per-instance quality measures to produce a plurality of adjusted per-instance quality; and
determine an overall quality score of the video asset based on the plurality of the adjusted per-instance quality measures wherein adjusting per-instance quality measures includes clipping the raw per-instance quality measures to a cap value that is computed as a function of the time-varying target quality.
18 . The system of claim 17 , wherein the computing device is further programmed to:
create a time-varying target quality curve based on user input with respect to the time-varying target quality at each of the plurality of time instances.
19 . The system of claim 17 , wherein the computing device is further programmed to, in assessing the video quality at each of the plurality of time instances, one or more of:
assess the video quality at a frame of the video asset corresponding to the respective time instance, or
assess the video quality of a time segment composed of multiple consecutive frames of the video asset near the respective time instance,
whereas to assess the video quality at each of the plurality of time instances further includes one or more of to:
conduct subjective testing,
compute an objective video quality metric, or
infer the video quality as a monotonically increasing function with respect to a video bit rate or a resolution of the video asset.
20 . The system of claim 17 , wherein the computing device is further programmed to, in computing the cap value as the function of the time-varying target quality, define the function as the time-varying target quality plus a delta value that is either constant or monotonically increasing with the time-varying target quality value.
21 . The system of claim 17 , wherein the computing device is further programmed to, in aggregating the adjusted per-instance quality measures into an overall quality assessment of the video asset, compute one or more of an average, percentile average, weighted average, or median of the adjusted per-instance quality measures.
22 . The system of claim 17 , wherein the computing device is further programmed to, in aggregating the adjusted per-instance quality measures into an overall quality assessment of the video asset:
extract a time window from the video asset;
compute a quality assessment of the time window as an average, a percentile average, a weighted average, or a median of the adjusted per-instance quality measures over the time window;
move starting and ending points of the time window over time; and
create a time varying quality curve by the quality assessment of the time window as moved.
23 . The system of claim 17 , wherein the computing device is further programmed to:
encode the video asset using a coding configuration that maximizes overall quality of the video asset.
24 . The system of claim 23 , wherein the computing device is further programmed to:
segment the video asset into time windows;
encode the video asset within each time window of the time windows using a coding configuration that satisfies target quality requirements of the plurality of time instances within the time window and minimizes a bit rate of the time window; and
connect the encodes of the time windows into a full encoding of the video asset.
25 . The system of claim 23 , wherein the computing device is further programmed to:
segment the video asset into time windows;
encode the video asset within each time window of the time windows using a coding configuration that maximizes a joint optimization objective function determined based on an average bit rate and a quality assessment of the time window under the time-varying target quality; and
connect the encodes of the time windows into a full encoding of the video asset.
26 . The system of claim 25 , wherein the computing device is further programmed to compute the joint optimization objective function as a sum of:
the overall quality of a video asset or the quality assessment of a time window; and
a product of the average bit rate of the video asset and a Lagrange parameter.
27 . The system of claim 23 , wherein the computing device is further programmed to define the coding configuration as a collection of a coding parameter P assigned to each of the time instances, the coding parameter P including one or more of a quantization parameter, a quality control parameter, or a bit rate.
28 . The system of claim 27 , wherein the computing device is further programmed to, in determining the coding parameter P:
compute a derivative of the overall quality of the video asset with respect to the coding parameter P;
compute a derivative of an average bit rate of the video asset with respect to the coding parameter P;
determine an optimal Lagrange parameter based on a ratio between the derivative of the overall quality of the video asset with respect to the coding parameter P and the derivative of the average bit rate of the video asset with respect to the coding parameter P; and
determine the coding parameter P using the optimal Lagrange parameter.
29 . The system of claim 17 , wherein the cap value specifies a constant maximum value for the raw per-instance quality measures below a maximum possible quality score, such that any raw per-instance quality measures exceeding the maximum possible quality score are reduced to the constant maximum value in computing the overall quality score of the video asset.