Method and system with image super-resolution
A method and system for screening a plurality of images for performing super-resolution (SR) are provided. The method includes using a multitask learning model (MLM) determining at least one of a plurality of image properties related to each of a plurality of images having a resolution lower than a predefined threshold; and selecting, based on the at least one of the plurality of image properties, a first set of images among the plurality of images to each be respectively suitable for upscaling; recommending, using a recommendation model, at least one parameter based on the at least one of the plurality of image properties; and generating at least one super-resolution image by respectively performing a super-resolution upscaling operation on at least one image of the first set of images based on the at least one parameter and the at least one of the plurality of image properties.
1 . A processor-implemented method, the method comprising:
a multitask learning model (MLM):
determining at least one of a plurality of image properties related to each of a plurality of images having a resolution lower than a predefined threshold; and
selecting, based on the at least one of the plurality of image properties, a first set of images among the plurality of images to each be respectively suitable for upscaling;
recommending, using a recommendation model, at least one parameter based on the at least one of the plurality of image properties; and
generating at least one super-resolution image by respectively performing a super-resolution upscaling operation on at least one image of the first set of images based on the at least one parameter and the at least one of the plurality of image properties.
2 . The method of claim 1 , wherein the plurality of image properties comprises image scale information, defect information, defect classification, a noise quotient, and a region of interest (ROI).
3 . The method of claim 1 , wherein the selecting further comprises respectively selecting each image of the first set of images in response to a corresponding value of the at least one of the plurality of image properties being greater than a predetermined threshold.
4 . The method of claim 1 , wherein the at least one parameter comprises an upscale factor.
5 . The method of claim 1 , further comprising:
training the MLM using a training dataset comprising the plurality of image properties and a set of low-resolution test images.
6 . The method of claim 1 ,
wherein the recommending using the recommendation model further comprises:
recommending at least one image analysis model based on the at least one of the plurality of image properties; and
recommending the at least one parameter by performing in-depth analysis on the at least one image of the first set of images or the generated at least one super-resolution image, using the recommended at least one image analysis model to determine a characteristic of an object in the at least one image of the first set of images.
7 . A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .
8 . An electronic device comprising:
one or more processors configured to execute instructions; and
one or more memoiesy configured to store the instructions,
wherein the execution of the one or more processors configures the one or more processors to:
determine at least one of a plurality of image properties related to each of a plurality of captured images having a resolution lower than a predefined threshold;
select a first set of images from the plurality of images based on the at least one of the plurality of image properties;
recommend at least one image analysis model and at least one parameter, based on the at least one of the plurality of image properties; and
generate super-resolution images by respectively performing a super-resolution unscaling operation on only the first set of images, where the first set of images includes less than all of the plurality of images, based on the at least one parameter and the at least one of the plurality of image properties.
9 . The electronic device of claim 8 , wherein the one or more processors are further configured to respectively select each image of the first set of images in response to a corresponding value of the at least one of the plurality of image properties being greater than a predetermined threshold.
10 . The electronic device of claim 8 , wherein the at least one parameter comprises an upscale factor.
11 . The electronic device of claim 8 , wherein the one or more processors are further configured to train a multitask learning model (MLM) using a training dataset comprising the plurality of image properties and a set of low-resolution test images.
12 . The electronic device of claim 8 , wherein the one or more processors are further configured to:
recommend at least one image analysis model based on the at least one of the plurality of image properties; and
recommend the at least one parameter by performing in-depth analysis on the at least one image of the first set of images or the generated at least one super-resolution image, using the recommended at least one image analysis model to determine a characteristic of an object in the at least one image of the first set of images.
13 . A system for screening a plurality of images for performing super-resolution, the system comprising:
one or more processors configured to execute instructions; and
one or more memories storing the instructions,
wherein the execution of the instructions configures the one or more processors to: use a multitask learning model (MLM):
determine at least one of a plurality of image properties related to each of a plurality of images having a resolution lower than a predefined threshold; and
select, based on the at least one of the plurality of image properties, a first set of images among the plurality of images to be respectively suitable for upscaling;
recommend, using a recommendation model, at least one parameter based on the at least one of the plurality of image properties; and
generate, using a super-resolution model, at least one super-resolution image respectively through performance of a super-resolution upscaling operation on at least one image among the first set of images based on the at least one parameter and the at least one of the plurality of image properties.
14 . The system of claim 13 , wherein the plurality of image properties comprises image scale information, defect information, defect classification, a noise quotient, and a region of interest (ROI).
15 . The system of claim 13 , wherein the one or more processors are further configured to respectively select each image of the first set of images in response to a corresponding value of the at least one of the plurality of image properties being greater than a predetermined threshold.
16 . The system of claim 13 , wherein the at least one parameter comprises an upscale factor.
17 . The system of claim 13 , further comprising:
a training model configured to train the MLM using a training dataset comprising the plurality of image properties and a set of low-resolution test images.
18 . The system of claim 13 , wherein the one or more processors are further configured to:
recommend at least one image analysis model based on the at least one of the plurality of image properties; and
recommend the at least one parameter by performing in-depth analysis on at least one image of the first set of images or the generated at least one super-resolution image, using the recommended at least one image analysis model to determine a characteristic of an object in the at least one image of the first set of images.
19 . The electronic device of claim 8 , wherein the plurality of image properties comprises image scale information, defect information, defect classification, a noise quotient, and a region of interest (ROI).