IP Library Granted Patent US 11,475,540
Granted Patent B2
US 11,475,540 · App. 16/830,943 · Granted Oct 18, 2022

Electronic device, control method thereof, and system

Inventors: Sangkwon Na (Suwon-si, KR); Wooseok Kang (Suwon-si, KR); Minho Kim (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06T3/4046G06N3/04G06N3/08G06T9/002
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,475,540
App. No.
16/830,943
Granted
Oct 18, 2022
Kind
B2
Abstract

An electronic device is provided. The electronic device includes: a memory configured to include at least one instruction; and a processor configured to be connected to the memory to control the electronic device, and obtain an output image by upscaling an input image using an artificial intelligence model trained to upscale an image, wherein the processor is configured to control the electronic device to: obtain parameter information of the artificial intelligence model based on pre-processing related information performed on the input image, and upscale the input image using the artificial intelligence model corresponding to the obtained parameter information.

Claims (37)

1. An electronic device comprising:

a memory configured to include at least one instruction;

a communication interface configured to receive, as an input image, from an external electronic device, an encoded downscaled image corresponding to an original image; and

a processor connected to the memory and configured to control the electronic device, and, based on receiving the input image and pre-processing related information related to a pre-processing performed by the external electronic device on the downscaled image after downscaling and prior to encoding, obtain an output image by upscaling the input image using an artificial intelligence model trained to upscale an image,

wherein the processor is configured to:

decode the input image,

obtain parameter information based on the pre-processing related information,

obtain the artificial intelligence model by setting a parameter corresponding to each of a plurality of neural network layers forming the artificial intelligence model based on the obtained parameter information, and

upscale the decoded image using the artificial intelligence model,

wherein the pre-processing related information received together with the input image includes information related to a filter applied to the downscaled image after downscaling of the original image and prior to encoding and information on a difference between the downscaled image before the filter is applied and the downscaled image after the filter is applied.

2. The electronic device as claimed in claim 1 , wherein the memory stores parameter information mapped to the pre-processing related information corresponding to each of different pre-processings, and

the processor is configured to identify parameter information corresponding to the received pre-processing related information among the stored parameter information, and upscale the input image using the artificial intelligence model based on the identified parameter information.

3. The electronic device as claimed in claim 2 , wherein the parameter information stored in the memory includes filter related information applied to each of the different pre-processings, and

the processor is configured to identify parameter information corresponding to the filter related information based on the received pre-processing related information among the stored parameter information, and upscale the input image using the artificial intelligence model corresponding to the identified parameter information.

4. The electronic device as claimed in claim 3 , wherein the filter related information applied to each of the different pre-processings includes information on at least one of a type or an intensity of a filter applied to each of the different pre-processings.

5. A method of controlling an electronic device, the method comprising:

based on receiving, as an input image, from an external electronic device, an encoded downscaled image corresponding to an original image and pre-processing related information related to a pre-processing performed by the external electronic device on the downscaled image after downscaling of the original image and prior to encoding, decoding the input image;

obtaining, based on the pre-processing related information, parameter information of an artificial intelligence model trained to upscale an image;

obtaining the artificial intelligence model by setting a parameter corresponding to each of a plurality of neural network layers forming the artificial intelligence model based on the obtained parameter information;

obtaining an output image by upscaling the decoded image using the artificial intelligence model,

wherein the pre-processing related information received together with the input image includes information related to a filter applied to the downscaled image after downscaling of the original image and prior to encoding and information on a difference between the downscaled image before the filter is applied and the downscaled image after the filter is applied.

6. The method as claimed in claim 5 , further comprising identifying parameter information corresponding to the received pre-processing related information in a memory storing parameter information mapped to pre-processing related information corresponding to each of different pre-processings, and

upscaling the input image using the artificial intelligence model based on the identified parameter information.

7. The method as claimed in claim 6 , wherein the parameter information stored in the memory includes filter related information applied to each of the different pre-processings, and the method further comprises:

identifying parameter information corresponding to the filter related information based on the received pre-processing related information among the stored parameter information, and

the obtaining of the output image includes upscaling the input image using the artificial intelligence model corresponding to the identified parameter information.

8. The method as claimed in claim 7 , wherein the filter related information applied to each of the different pre-processings includes information on at least one of a type or an intensity of a filter applied to each of the different pre-processings.

9. An electronic device comprising:

a communication interface;

a memory configured to store parameter information mapped to pre-processing related information related to each of different pre-processings performed by the external electronic device on a downscaled image after downscaling of an original image; and

a processor connected to the memory and configured to control the electronic device, and to obtain an output image by upscaling an input image using an artificial intelligence model trained to upscale an image,

wherein the processor is configured to:

based on receiving, as an input image, from an external electronic device via the communication interface, an encoded downscaled image corresponding to an original image and pre-processing related information related to a pre-processing performed by the external electronic device after downscaling of the original image and prior to encoding, decode the input image;

identify parameter information corresponding to the received pre-processed related information among the stored parameter information;

set a parameter of each of a plurality of neural network layers included in the artificial intelligence model based on the identified parameter information; and

upscale the decoded image using the artificial intelligence model having the set parameter for each neural network layer,

wherein the parameter information stored in the memory includes information on at least one of a type or an intensity of a filter for each of the different pre-processings and information on a difference between the downscaled image before the filter is applied and the downscaled image after the filter is applied.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 26, 2020
From: NA, SANGKWON; KANG, WOOSEOK; KIM, MINHO
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 052236/0456 →
Priority Claims (1)
KR 10-2019-0157745 · Nov 29, 2019 · national
Continuity (1)
Related Publication 20210166344A1 · Jun 3, 2021
Cited By (3)
US 12,223,617 US 12,412,242 US 12,542,917