IP Library › Granted Patent US 11,138,763
Granted Patent B2
US 11,138,763 · App. 17/082,848 · Granted Oct 5, 2021

Methods and apparatuses for performing artificial intelligence encoding and artificial intelligence decoding on image

Inventors: Jaehwan Kim (Suwon-si, KR); Jongseok Lee (Suwon-si, KR); Sunyoung Jeon (Suwon-si, KR); Kwangpyo Choi (Suwon-si, KR); Minseok Choi (Suwon-si, KR); Quockhanh Dinh (Suwon-si, KR); Youngo Park (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06T9/002G06T3/4046H04N19/85
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,138,763
App. No.
17/082,848
Granted
Oct 5, 2021
Kind
B2
Abstract

Provided is an artificial intelligence (AI) decoding apparatus includes: a memory storing one or more instructions; and a processor configured to execute the one or more instructions stored in the memory, the processor is configured to: obtain AI data related to AI down-scaling an original image to a first image; obtain image data corresponding to an encoding result on the first image; obtain a second image corresponding to the first image by performing a decoding on the image data; obtain deep neural network (DNN) setting information among a plurality of DNN setting information from the AI data; and obtain, by an up-scaling DNN, a third image by performing the AI up-scaling on the second image, the up-scaling DNN being configured with the obtained DNN setting information, wherein the plurality of DNN setting information comprises a parameter used in the up-scaling DNN, the parameter being obtained through joint training of the up-scaling DNN and a down-scaling DNN, and wherein the down-scaling DNN is used to obtain the first image from the original image.

Claims (32)

1. An electronic device for displaying an image by using artificial intelligence (AI), the electronic device comprising:

a display; and

one or more processors, when executing one or more instructions stored in the electronic device, configured to:

receive AI data related to AI down-scaling an original image to a first image through a down-scaling neural network (NN) based on first NN setting information, and an image data generated through an encoding on the first image, the first NN setting information being selected from a plurality of first NN setting information, and the AI data comprising a bitrate corresponding to the image data;

obtain a second image by decoding the image data;

select, based on the AI data, second NN setting information paired with the selected first NN setting information from a plurality of second NN setting information;

obtain, by an up-scaling NN, a third image by performing AI up-scaling on the obtained second image, the up-scaling NN being set with the selected second NN setting information; and

provide, on the display, the obtained third image,

wherein the plurality of first NN setting information and the plurality of second NN setting information are paired with each other.

2. The electronic device of claim 1 , wherein the AI data comprises information related to the first image,

wherein the one or more processors, when executing the one or more instructions, are further configured to select the second NN setting information mapped to the information related to the first image, based on a mapping relationship between a plurality of image-related information and the plurality of second NN setting information.

3. The electronic device of claim 1 , wherein the AI data further comprises at least one of a quantization parameter regarding the image data, a resolution of the first image, or a codec type used to encode the first image.

4. The electronic device of claim 1 , wherein the selected second NN setting information comprises neural network parameters for at least one convolution layer in the up-scaling NN.

5. A server for providing an image by using an artificial intelligence (AI), the server comprising:

one or more processors configured to execute one or more instructions stored in the server to:

select first neural network (NN) setting information from a plurality of first NN setting information;

obtain, by a down-scaling NN, a first image by performing AI down-scaling on an original image, the down-scaling NN being set with the selected first NN setting information;

encode the first image to obtain image data; and

provide an electronic device with the image data and AI data related to the AI down-scaling, the AI data comprising a bitrate corresponding to the image data and being used to select second NN setting information paired with the selected first NN setting information from a plurality of second NN setting information for AI up-scaling,

wherein the plurality of first NN setting information and the plurality of second NN setting information are paired with each other, and the AI up-scaling is performed by an up-scaling NN set with the selected second NN setting information.

6. The server of claim 5 , wherein the one or more processors is configured to select the first NN setting information from the plurality of first NN setting information based on a compression ratio input from a user.

7. The server of claim 5 , wherein the one or more processors is configured to select the first NN setting information from the plurality of first NN setting information based on compression history information.

8. The server of claim 5 , wherein the one or more processors is configured to select the first NN setting information from the plurality of first NN setting information based on at least one of a resolution of the original image or a type of the original image.

9. The server of claim 5 , wherein the AI data further comprises at least one of a quantization parameter regarding the image data, a resolution of the first image, or a codec type used to encode the first image.

10. A non-transitory computer-readable recording medium having recorded thereon instructions, which when executed by an electronic device, cause the electronic device to perform first operations comprising:

receiving artificial intelligence (AI) data related to AI down-scaling an original image to a first image through a down-scaling neural network (NN) based on first NN setting information, and image data corresponding to an encoding result on the first image, the first NN setting information being selected from a plurality of first NN setting information, and the AI data comprising a bitrate corresponding to the image data;

decoding the image data to obtain a second image; and

providing, on a display of the electronic device, a third image that is obtained by performing AI up-scaling on the second image, and

which when executed by the electronic device, causes the electronic device to perform second operations comprising:

selecting, based on the AI data, second NN setting information paired with the selected first NN setting information, from a plurality of second NN setting information for the AI up-scaling; and

obtaining, by an up-scaling NN, the third image by performing the AI up-scaling on the obtained second image, the up-scaling NN being set with the selected second NN setting information,

wherein the plurality of first NN setting information and the plurality of second NN setting information are paired with each other.

Priority Claims (3)
KR 10-2018-0125406 · Oct 19, 2018 · national
KR 10-2019-0053248 · May 7, 2019 · national
KR 10-2019-0062583 · May 28, 2019 · national
Continuity (3)
Continuation 16570057 · Sep 13, 2019
Continuation PCTKR2019004171 · Apr 8, 2019
Related Publication 20210118189A1 · Apr 22, 2021