Unsupervised super-resolution training data construction
A method for training a super-resolution network may include obtaining a low resolution image; generating, using a first machine learning model, a first high resolution image based on the low resolution image; generating, using a second machine learning model, a second high resolution image based on the first high resolution image and an unpaired dataset of high resolution images; obtaining a training data set using the low resolution image and the second high resolution image; and training the super-resolution network using the training data set.
1. A method for training a super-resolution network, the method comprising:
obtaining a low resolution image;
generating, using a first machine learning model, a first high resolution image based on the low resolution image;
generating, using a second machine learning model, a second high resolution image based on the first high resolution image and an unpaired dataset of high resolution images, by minimizing an adversarial loss, a cycle loss, and a low-frequency content preservation loss that is defined as a distance between an input and an output of the second machine learning model after passing the input and the output through a low-pass filter;
obtaining a training data set using the low resolution image and the second high resolution image; and
training the super-resolution network using the training data set.
2. The method of claim 1 , wherein the generating the first high resolution image comprises generating the first high resolution image by minimizing an image loss function and a kernel loss function.
3. The method of claim 1 , wherein the generating the first high resolution image comprises generating the first high resolution image by identifying a blur kernel.
4. The method of claim 1 , wherein the generating the second high resolution image comprises generating the second high resolution image by training a generator that performs domain adaptation between the first high resolution image generated by the first machine learning model and a real high resolution image.
5. The method of claim 1 , wherein the training the super-resolution network comprises training the super-resolution network to obtain an input low resolution image captured by a user device, and output a high resolution image.
6. A device for training a super-resolution network, the device comprising:
a memory configured to store instructions; and
a processor configured to execute the instructions to:
obtain a low resolution image;
generate, using a first machine learning model, a first high resolution image based on the low resolution image;
generate, using a second machine learning model, a second high resolution image based on the first high resolution image and an unpaired dataset of high resolution images, by minimizing an adversarial loss, a cycle loss, and a low-frequency content preservation loss that is defined as a distance between an input and an output of the second machine learning model after passing the input and the output through a low-pass filter;
obtain a training data set using the low resolution image and the second high resolution image; and
train the super-resolution network using the training data set.
7. The device of claim 6 , wherein the processor is configured to generate the first high resolution image by minimizing an image loss function and a kernel loss function.
8. The device of claim 6 , wherein the processor is configured to generate the first high resolution image by identifying a blur kernel.
9. The device of claim 6 , wherein the processor is configured to generate the second high resolution image by training a generator that performs domain adaptation between the first high resolution image generated by the first machine learning model and a real high resolution image.
10. The device of claim 6 , wherein the first machine learning model is a deep image prior.
11. The device of claim 6 , wherein the second machine learning model is a generative adversarial network (GAN).