Granted Patent B2

US 12,327,340 · App. 17/842,309 · Granted Jun 10, 2025

Inventors: Pawan Prasad Bindigan Hariprasanna (Bengaluru, IN); Saikat Kumar Das (Bengaluru, IN); Green Rosh K S (Bengaluru, IN); Lokesh Rayasandra Boregowda (Bengaluru, IN); Balvinder Singh (Bengaluru, IN); Venkat Ramana Peddigari (Bengaluru, IN); Alok Shankarlal Shukla (Bengaluru, IN)

Assignee: Samsung Electronics Co., Ltd.

G06T5/80G06T3/4046G06T5/20G06V10/764G06V10/776G06T2207/10016G06T2207/20016G06T2207/20081G06T2207/20084G06T2207/30242

Abstract

Claims (50)

1. A method for removing an artifact from a high resolution image by an electronic device, the method comprising:

receiving, by the electronic device, the high resolution image comprising the artifact;

downscaling, by the electronic device, the high resolution image into a plurality of lower resolution images;

removing, by the electronic device, the artifact from the plurality of lower resolution images by applying at least one first machine learning model from a plurality of machine learning models on the plurality of lower resolution images; and

generating, by the electronic device, a high resolution image without the artifact by applying at least one second machine learning model from the plurality of machine learning models on an output from the at least one first machine learning model,

wherein an output from each of the machine learning models comprises a low resolution image without the artifact.

2. The method of claim 1 , wherein the downscaling, by the electronic device, of the high resolution image into the plurality of lower resolution images comprises:

determining, by the electronic device, a number of scales of a Low scale sub-network (LSSNet) based on at least one of a predetermined lookup table, a user input, or a number of artifacts to be removed from the high resolution image, and a type of artifacts to be removed from the high resolution image; and

downscaling, by the electronic device, the high resolution image into the plurality of lower resolution images based on the number of scales of the LSSNet.

3. The method of claim 2 , wherein an iterative Progressive Inference (PI) scheme is used for estimating scales {2, . . . , N} after a first scale O 1 is estimated.

4. The method of claim 1 , wherein the removing, by the electronic device, of the artifact from the plurality of lower resolution images by applying the at least one first machine learning model on the plurality of lower resolution images comprises:

determining, by the electronic device, a number of artifacts to be removed from the high resolution image;

determining, by the electronic device, a type of each of the artifacts to be removed from the high resolution image;

prioritizing, by the electronic device, the number of artifacts to be removed in a sequence based on the type of each of the artifacts to be removed from the high resolution image;

determining, by the electronic device, a sequence of the at least one first machine learning model of the plurality of machine learning models for removing the artifact from the high resolution image based on the prioritized number of artifacts;

feeding, by the electronic device, each lower resolution image from the plurality of lower resolution images into the at least one first machine learning model based on the determined sequence; and

removing, by the electronic device, the artifact from each of the lower resolution images using the at least one first machine learning model.

5. The method of claim 1 , wherein the generating, by the electronic device, of the high resolution image without the artifact by applying the at least one second machine learning model from the plurality of machine learning models on an output from the at least one first machine learning model comprises:

obtaining, by the electronic device, the output comprising the low resolution image without the artifact from the at least one machine learning model;

generating, by the electronic device, high resolution output by upscaling the low resolution image without the artifact using a Convolutional Guided Filter (CGF) of at least one second machine learning model from the plurality of machine learning models, wherein the CGF upscales the low resolution image without the artifact using the higher resolution as a guide; and

generating, by the electronic device, the high resolution image without the artifact by passing the high resolution output from a High Scale Sub-network (HSSNet).

6. The method of claim 5 , wherein the CGF operates at multiple pyramid levels with each pyramid level comprising an identical deep learning model with weights and bias being shared across the pyramid levels.

7. The method of claim 5 , wherein both the sub networks LSSNet and HSSNet are trained using a combination of different loss functions comprising pixel and feature losses.

8. The method of claim 1 , wherein a receptive field is set dynamically by changing number of scale-space levels.

9. An electronic device for removing an artifact from a high resolution image, wherein the electronic device comprises:

a memory;

a processor; and

an artifact removal controller, communicatively connected to the memory and the processor,

wherein the artifact removal controller is configured to:

receive the high resolution image comprising the artifact,

downscale the high resolution image into a plurality of lower resolution images,

remove the artifact from the plurality of lower resolution images by

applying at least one first machine learning model from a plurality of

machine learning models on the plurality of lower resolution images, and generate a high resolution image without the artifact by applying at least one second machine learning model from the plurality of machine learning models on an output from the at least one first machine learning model,

wherein an output from each of the machine learning models comprises a low resolution image without the artifact.

10. The electronic device of claim 9 , wherein downscaling the high resolution image into the plurality of lower resolution images comprises:

determining a number of scales of a Low scale sub-network (LSSNet) based on at least one of a predetermined lookup table, a user input, or a number of artifacts to be removed from the high resolution image, and a type of artifacts to be removed from the high resolution image; and

downscaling the high resolution image into the plurality of lower resolution images based on the number of scales of the LSSNet.

11. The electronic device of claim 9 , wherein removing the artifact from the plurality of lower resolution images by applying the at least one first machine learning model on the plurality of lower resolution images comprises:

determining a number of artifacts to be removed from the high resolution image;

determining a type of each of the artifacts to be removed from the high resolution image;

prioritizing the number of artifacts to be removed in a sequence based on the type of each of the artifacts to be removed from the high resolution image;

determining a sequence of the at least one first machine learning model of the plurality of machine learning models for removing the artifact from the high resolution image based on the prioritized number of artifacts;

feeding each lower resolution image from the plurality of lower resolution images into the at least one first machine learning model based on the determined sequence; and

removing the artifact from each of the lower resolution images using the at least one first machine learning model.

12. The electronic device of claim 9 , wherein generating the high resolution image without the artifact by applying the at least one second machine learning model from the plurality of machine learning models on an output from the at least one first machine learning model comprises:

obtaining the output comprising the low resolution image without the artifact from the at least one machine learning model;

generating a high resolution output by upscaling the low resolution image without the artifact by using a Convolutional Guided Filter (CGF) of at least one second machine learning model from the plurality of machine learning models, wherein the CGF upscales the low resolution image without the artifact by using the higher resolution as a guide; and

generating the high resolution image without the artifact by passing the high resolution output from a High Scale Sub-network (HSSNet).

13. The electronic device of claim 12 , wherein the CGF operates at multiple pyramid levels with each pyramid level comprising an identical deep learning model with weights and bias being shared across the pyramid levels.

Assignments (1)

Priority Claims (2)

Continuity (2)

References Cited (29)