IP Library › Granted Patent US 11,954,833
Granted Patent B2
US 11,954,833 · App. 17/678,557 · Granted Apr 9, 2024

Electronic device for supporting machine learning-based image processing

Inventors: Woojhon Choi (Gyeonggi-do, KR); Wonjoon Do (Gyeonggi-do, KR); Jaesung Choi (Gyeonggi-do, KR)
Assignee: Samsung Electronics Co., Ltd
G06T5/50H04N23/815G06T2207/10024G06T2207/20081G06T2207/20224
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,954,833
App. No.
17/678,557
Granted
Apr 9, 2024
Kind
B2
Abstract

An electronic device is provided, which includes a display; an under display camera (UDC); a processor; and a memory that stores instructions, which when executed, causes the processor to receive a first image from the UDC, generate a second image by lowering a resolution of the first image, obtain a third image representing a difference between the second image and an image corresponding to the second image in which artifacts are not present, generate a fourth image by increasing the resolution of the third image, and generate a fifth image by subtracting the fourth image from the first image.

Claims (46)

1. An electronic device, comprising:

a camera;

a display located between a subject to be photographed by the camera and the camera;

a processor connected to the camera and the display; and

a memory that stores instructions, which when executed, cause the processor to:

receive a first image from the camera,

generate a second image by lowering a resolution of the first image from NM to (N/n)(M/m),

put the second image into an artificial intelligence (AI) model learned using a first training image having a resolution of (N/n)(M/m) and a second training image having a resolution of (N/n)(M/m) and making a pair with the first training image as an input value,

obtain a third image from a resultant value output from the AI model representing a difference between the second image and an image corresponding to the second image in which artifacts are not present,

generate a fourth image by increasing a resolution of the third image from (N/n)*(M/m) to N*M, and

generate a fifth image by subtracting the fourth image from the first image.

2. The electronic device of claim 1 , wherein the instructions, when executed, further cause the processor to:

transmit the second image to the AI model provided in an external electronic device through a wireless communication circuit of the electronic device, and

obtain the third image from the AI model of the external electronic device through the wireless communication circuit.

3. The electronic device of claim 1 , wherein the instructions, when executed, further cause the processor to receive data including a Bayer pattern or a red-green-blue (RGB) pattern from the camera as the first image.

4. The electronic device of claim 1 , wherein the camera obtains data including a Bayer pattern or a red-green-blue (RGB) pattern by photographing the subject, and processes the data, and

wherein the instructions further cause the processor to receive the processed data from the camera as the first image.

5. The electronic device of claim 1 , wherein the instructions, when executed, further cause the processor to display the fifth image or a sixth image having a resolution that is lower than the resolution of the fifth image on the display.

6. The electronic device of claim 1 , wherein the instructions, when executed, further cause the processor to perform at least one operation of storing the fifth image in the memory and transmitting the fifth image to an external electronic device through a wireless communication circuit of the electronic device.

7. A method performed by an electronic device, the method comprising:

receiving a first image from a camera of the electronic device;

generating a second image by lowering a resolution of the first image from N*M to (N/n)*(M/m);

putting the second image into an artificial intelligence (AI) model learned using a first training image having a resolution of (N/n)*(M/m) and a second training image having a resolution of (N/n)*(M/m) and making a pair with the first training image as an input value;

obtaining a third image from a resultant value output from the AI model, the third image representing a difference between the second image and an image corresponding to the second image in which artifacts are not present;

generating a fourth image by increasing a resolution of the third image from (N/n)*(M/m) to N*M; and

generating a fifth image by subtracting the fourth image from the first image.

8. The method of claim 7 , wherein obtaining the third image comprises:

transmitting the second image to the AI model provided in an external electronic device through a wireless communication circuit of the electronic device; and

obtaining the third image from the AI model of the external electronic device through the wireless communication circuit.

9. The method of claim 7 , wherein receiving the first image comprises receiving data including a Bayer pattern or a red-green-blue (RGB) pattern from the camera as the first image.

10. The method of claim 7 , wherein the camera obtains data including a Bayer pattern or a red-green-blue (RGB) pattern by photographing the subject, and processes the data, and

wherein receiving the first image comprises receiving the processed data from the camera as the first image.

11. The method of claim 7 , further comprising displaying the fifth image or a sixth image having a resolution that is lower than the resolution of the fifth image on a display of the electronic device.

12. The method of claim 7 , further comprising:

performing at least one operation of storing the fifth image in a memory of the electronic device; and

transmitting the fifth image to an external electronic device through a wireless communication circuit of the electronic device.

13. A non-transitory recording medium that stores instructions, which when executed by a processor of an electronic device, cause the processor to:

receive a first image from a camera of the electronic device;

generate a second image by lowering a resolution of the first image from N*M to (N/n)*(M/m);

put the second image into an artificial intelligence (AI) model learned using a first training image having a resolution of (N/n)*(M/m) and a second training image having a resolution of (N/n)*(M/m) and making a pair with the first training image as an input value;

obtain a third image from a resultant value output from the AI model, the third image representing a difference between the second image and an image corresponding to the second image in which artifacts are not present;

generate a fourth image by increasing a resolution of the third image from (N/n)*(M/m) to N*M; and

generate a fifth image by subtracting the fourth image from the first image.

14. The non-transitory recording medium of claim 13 , wherein receiving the first image comprises receiving data including a Bayer pattern or a red-green-blue (RGB) pattern from the camera as the first image.

15. The non-transitory recording medium of claim 13 , wherein the camera obtains data composed of a Bayer pattern or a red-green-blue (RGB) pattern by photographing the subject, and processes the data, and

wherein receiving the first image comprises receiving the processed data from the camera as the first image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 28, 2022
From: CHOI, WOOJHON; DO, WONJOON; CHOI, JAESUNG
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 059114/0795 →
Priority Claims (1)
KR 10-2021-0043585 · Apr 2, 2021 · national
Continuity (2)
Continuation PCTKR2022001732 · Feb 4, 2022
Related Publication 20220318971A1 · Oct 6, 2022