Method and device for evaluating subjective quality of video
Proposed are a method and apparatus for evaluating the quality of an image, the method including obtaining blocks each having a predetermined size by splitting a target image for evaluating a quality and a reference image that is to be compared with the target image, determining sensitivity information and quality assessment information of each of the blocks by inputting the blocks to a video quality assessment network, and determining a final image quality assessment score of the target image by combining the pieces of quality assessment information of the blocks with each other, based on the pieces of sensitivity information of the blocks.
1. A method of evaluating a quality of a target image, the method comprising:
obtaining blocks each having a predetermined size by splitting a target image for evaluating a quality and a reference image that is to be compared with the target image;
determining pieces of sensitivity information and pieces of quality assessment information of the blocks by inputting the blocks to a video quality assessment network; and
determining a final image quality assessment score of the target image by combining the pieces of quality assessment information of the blocks with each other, based on the pieces of sensitivity information of the blocks.
2. The method of claim 1 , wherein the video quality assessment network is configured to:
determine the pieces of sensitivity information of the blocks by inputting the blocks to a convolutional neural network, and
determine the pieces of quality assessment information of the blocks by inputting the blocks to a capsule neural network.
3. The method of claim 1 , wherein the sensitivity information is used as a weight of each of the blocks,
wherein the quality assessment information comprises a mean opinion score (MOS) average of each block of the blocks, and
wherein the final image quality assessment score is further determined based on a weighted average of the MOS average for the each block based on the weight, and
wherein the MOS is a value representing a user's subjective perceived quality.
4. The method of claim 1 , wherein the video quality assessment network is configured to:
determine pieces of sensitivity information of blocks of a target training image and a reference training image by inputting the blocks of the target training image and the reference training image to a convolutional neural network, and
determine mean opinion score (MOS) averages and MOS standard deviations of the blocks of the target training image and the blocks of the reference training image by inputting the blocks of the target training image and the blocks of the reference training image to a capsule neural network, and
wherein the video quality assessment network is trained using the pieces of sensitivity information, the MOS averages, and the MOS standard deviations.
5. The method of claim 1 , wherein the video quality assessment network is configured to:
determine pieces of sensitivity information of blocks of a target training image and a reference training image by inputting the blocks of the target training image and the reference training image to a convolutional neural network,
determine pieces of full reference quality assessment information of the blocks of the target training image and the reference training image by inputting the blocks of the target training image and the reference training image to a capsule neural network, and
determine pieces of non- reference quality assessment information of blocks of the target training image by inputting the blocks of the target training image to the capsule neural network, and
wherein the video quality assessment network is trained using the pieces of sensitivity information, the full reference quality assessment information, and the non-reference quality assessment information.
6. The method of claim 5 , wherein the full reference quality assessment information comprises a full reference mean opinion score (MOS) average and a full reference MOS standard deviation of each block of the blocks of the target training image and the reference training image,
wherein the non-reference quality assessment information comprises a non-reference MOS average and a non-reference MOS standard deviation of each block of the blocks of the target training image, and
wherein the MOS is a value representing a user's subjective perceived quality.
7. The method of claim 1 , wherein the predetermined size is txbxbxc, where t indicates the number of frames, b indicates a horizontal or vertical size, and c indicates the number of color channels.
8. The method of claim 1 , wherein the quality assessment information comprises a mean opinion score (MOS) average and a MOS standard deviation of each block of the blocks,
wherein the sensitivity information is used as a weight of the each block,
wherein the final image quality assessment score is further determined based on a weighted average of the MOS average based on the weight and a weighted average of the MOS standard deviation based on the weight, and
wherein the MOS is a value representing a user's subjective perceived quality.
9. The method of claim 1 , wherein the combining the pieces of quality assessment information of the blocks with each other comprises determining a weighted average of the pieces of quality information, and
wherein the weighted average is determined by using the pieces of sensitivity information as weights.