Method for evaluating video quality based on non-reference video
A method for evaluating video quality based on a non-reference video to evaluate the quality of a received video using artificial intelligence composed of convolutional neural networks (CNN) and recurrent neural networks (RNN) capable of setting a learning range, includes: (a) extracting RGB values for one frame of a video; (b) obtaining an output by providing the extracted RGB values to CNN No. 1; (c) obtaining an output by providing the extracted RGB values n times to a CNN; (d) repeating (a) to (c) for all frames and merging outputs of all CNNs; (e) obtaining the output of the RNN with the time dimension reduced to 1 after passing the merged output value to the RNN; and (f) after applying a regression algorithm to the final output of the RNN, predicting this value as video quality value.
1 . A method for evaluating video quality based on a non-reference video, the method comprising:
Operation (a) of extracting RGB values for one of a plurality of frames of a video;
Operation (b) of obtaining an output value by providing the extracted RGB values to a convolutional neural network (CNN) No. 1;
Operation (c) of obtaining an output by providing the extracted RGB values n times (n≥an integer greater than or equal to 2) to a CNN No. 2;
Operation (d) of repeating Operations (a) to (c) for all the frames of the video and merging outputs of CNN No. 1 and CNN No. 2 to generate merged output, the merged output having a frame number dimension and one or more other data dimensions;
Operation (e) comprising:
changing, via global average pooling, the one or more other data dimensions of the merged output to the sum of (1) the number of filters of CNN No. 1 and (2) the number of filters of CNN No. 2, to generate modified merged output, and
obtaining an output of a recurrent neural network (RNN) with a time dimension reduced to 1 after passing the modified merged output to the RNN for learning in the time dimension; and
Operation (f) of predicting a video quality value, after applying a regression algorithm to the output of the RNN to obtain a one-dimensional output value.
2 . The method of claim 1 , wherein:
the CNN No. 1 comprises a plurality of convolutional layers and wherein some of the convolutional layers of the CNN No. 1 are disabled for learning, while other ones of the convolutional layers of the CNN No. 1 are enabled for learning.
3 . The method of claim 2 , wherein:
certain ones of the convolutional layers of CNN No. 1 are pre-learned with a first plurality of image learning data and have a first plurality of fixed coefficients, and wherein remaining ones of the convolutional layers of CNN No. 1 start learning from the first plurality of coefficients, and wherein the first plurality of image learning data comprises ImageNet learning data.
4 . The method of claim 3 , wherein:
the n is 2, and
the CNN No. 2 comprises a plurality of convolutional layers and wherein some of the convolutional layers of CNN No. 2 are pre-learned with a second plurality of image learning data different from the first plurality of image learning data and have a second plurality of fixed coefficients, and the remaining ones of the convolutional layers of CNN No. 2 start learning from the second plurality of coefficients.
5 . The method of claim 3 , further comprising additional learning of CNN No. 1.
6 . The method of claim 4 , wherein:
an error backpropagation is used in the learning of the RNN and non-frozen layers of CNN No. 1 and CNN No. 2 in Operation (e).
7 . The method of claim 4 , further comprising additional learning of CNN No. 2.