Method and apparatus with region of interest extraction
An apparatus with region of interest (ROI) extraction includes: a processor configured to: generate an input image by distorting an original image comprising one or more objects; determine, based on the original image, a quality score of the input image using a machine learning model that is trained based on a mean opinion score (MOS) dataset; generate a class activation map for the input image based on the quality score of the input image; and extract an ROI from the original image based on the class activation map.
1 . An apparatus with region of interest (ROI) extraction, the apparatus comprising:
one or more processors; and
a memory comprising one or more non-transitory storage media that store instructions that, when executed by the one or more processors, configures the apparatus to:
generate an input image by distorting an original image comprising one or more objects;
determine, based on the original image, a quality score of the input image using a machine learning model that is trained based on a mean opinion score (MOS) dataset;
generate a class activation map for the input image based on the quality score of the input image; and
extract an ROI from the original image based on the class activation map.
2 . The apparatus of claim 1 , wherein, for the extracting of the ROI, the execution of the instructions further configures the apparatus to extract the ROI by binarizing the class activation map.
3 . The apparatus of claim 1 , wherein
the class activation map comprises a plurality of pixels,
one or more portions of the plurality of pixels has different pixel values, and
for the extracting of the ROI, the execution of the instructions further configures the apparatus to extract the ROI from the original image based the different pixel values of the plurality of pixels of the class activation map and a threshold value.
4 . The apparatus of claim 3 , wherein, for the extracting of the ROI, the execution of the instructions further configures the apparatus to determine pixels of the plurality of pixels of the class activation map having pixel values higher than the threshold value as included in the ROI.
5 . The apparatus of claim 1 , wherein, for the generating of the input image, the execution of the instructions further configures the apparatus to generate the input image by blurring the original image.
6 . The apparatus of claim 1 , wherein the execution of the instructions further configures the apparatus to:
for the generating of the input image, generate a plurality of different input images by distorting the original image using a plurality of different random masks;
for the determining of the quality score, determine a quality score of each of the plurality of different input images; and
for the extracting of the ROI, extract the ROI based on the quality score of each of the plurality of different input images.
7 . The apparatus of claim 6 , wherein
the plurality of different random masks is configured to distort different regions of the original image, and
the execution of the instructions further configures the apparatus to:
determine a quality score of the original image; and
for the extracting of the ROI, extract the ROI based on a difference between the quality score of each of the plurality of different input images and the quality score of the original image.
8 . The apparatus of claim 1 , wherein, for the determining of the quality score, the execution of the instructions further configures the apparatus to input the original image and the input image as an input to the machine learning model.
9 . The apparatus of claim 1 , wherein the execution of the instructions further configures the apparatus to:
quantize pixel values of pixels included in the ROI and convert the quantized pixel values into a block-based data map; and
generate a compressed image in which the original image is compressed, based on the block-based data map.
10 . An apparatus with region of interest (ROI) extraction, the apparatus comprising:
one or more processors; and
a memory comprising one or more non-transitory storage media that store instructions that, when executed by the one or more processors, configures the apparatus to:
determine a quality score of an input image using a machine learning model that is trained based on a mean opinion score (MOS) dataset;
generate a class activation map for the input image based on the quality score of the input image; and
extract an ROI from an original image of the input image based on the class activation map.
11 . The apparatus of claim 10 , wherein, for the extracting of the ROI, the execution of the instructions further configures the apparatus to extract the ROI by binarizing the class activation map.
12 . The apparatus of claim 10 , wherein
the class activation map comprises a plurality of pixels,
one or more portions of the plurality of pixels has different pixel values, and
for the extracting of the ROI, the execution of the instructions further configures the apparatus to extract the ROI from the original image based on the different pixel values of the plurality of pixels of the class activation map and a threshold value.
13 . The apparatus of claim 10 , wherein the quality score of the input image is determined by the machine learning model using only the input image without the original image.
14 . A processor-implemented method with region of interest (ROI) extraction, the method comprising:
determining a quality score of an input image distorted from an original image;
comprising one or more objects, using a machine learning model trained based on a mean opinion score (MOS) dataset;
generating a class activation map for the input image based on the quality score of the input image; and
extracting an ROI from the original image based on the class activation map.
15 . The method of claim 14 , wherein the extracting of the ROI from the original image based on the class activation map comprises extracting the ROI by binarizing the class activation map.
16 . The method of claim 14 , wherein
the class activation map comprises a plurality of pixels,
one or more portions of the plurality of pixels has different pixel values, and
the extracting of the ROI from the original image based on the class activation map comprises extracting the ROI from the original image based the different pixel values of the plurality of pixels of the class activation map and a threshold value.
17 . The method of claim 14 , further comprising generating the input image by distorting the original image,
wherein the determining of the quality score of the input image comprises determining the quality score of the input image based on the original image.
18 . The method of claim 14 , further comprising generating a plurality of different input images by distorting the original image using a plurality of different random masks,
wherein the determining of the quality score of the input image comprises determining quality scores of each of the plurality of different input images, and
wherein the extracting of the ROI comprises extracting the ROI from the original image based on the quality scores of each of the plurality of different input images.
19 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 14 .