IP Library Granted Patent US 12,229,918
Granted Patent B2
US 12,229,918 · App. 17/748,723 · Granted Feb 18, 2025

Electronic device for sharpening image and operation method thereof

Inventors: Pawel Kies (Warsaw, PL); Radoslaw Chmielewski (Warsaw, PL)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06T5/20G06T5/73H04N23/80
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,229,918
App. No.
17/748,723
Granted
Feb 18, 2025
Kind
B2
Abstract

Provided are an electronic device for sharpening an image and an operation method thereof. A method, performed by the electronic device, of sharpening an image includes: obtaining an image generated by a camera of the electronic device; obtaining a first sharpening kernel for enhancing sharpness of the image, wherein the first sharpening kernel is data including a plurality of weights to be applied to pixels in the image, the data being of a lower resolution than the image; determining coordinates corresponding to some weights indicating representative values of the first sharpening kernel from among the plurality of weights in the first sharpening kernel; generating a second sharpening kernel by selecting the weights corresponding to the determined coordinates; and obtaining a sharpened image by applying the second sharpening kernel to the image.

Claims (52)

1. A method, performed by an electronic device, the method comprising:

obtaining an image by a camera of the electronic device;

obtaining a first sharpening kernel for enhancing sharpness of the image, wherein the first sharpening kernel is data including a plurality of weights to be applied to pixels in the image, and the data is of a lower resolution than the image;

determining coordinates corresponding to first weights indicating representative values of the first sharpening kernel from among the plurality of weights in the first sharpening kernel, wherein the representative values are values indicating feature points of a shape of the first sharpening kernel;

generating a second sharpening kernel by selecting the first weights corresponding to the determined coordinates; and

obtaining a sharpened image by applying the second sharpening kernel to the image.

2. The method of claim 1 , wherein the obtaining the first sharpening kernel comprises:

obtaining, by using the camera of the electronic device, a reference image;

calculating a point spread function based on the reference image; and

obtaining the first sharpening kernel based on the point spread function.

3. The method of claim 1 , wherein the determining the coordinates corresponding to the first weights indicating the representative values of the first sharpening kernel comprises determining as zero point coordinates of a weight, from among the plurality of weights, corresponding to a center point of the first sharpening kernel.

4. The method of claim 1 , further comprising obtaining an augmented first sharpening kernel including new weights by performing interpolation between adjacent weights among the plurality of weights in the first sharpening kernel.

5. The method of claim 4 , wherein the determining the coordinates corresponding to the first weights comprises determining the coordinates corresponding to the first weights from among the new weights in the augmented first sharpening kernel.

6. The method of claim 5 , further comprising:

obtaining a third sharpening kernel by selecting second weights in the augmented first sharpening kernel; and

obtaining a fourth sharpening kernel by selecting third weights from the augmented first sharpening kernel at certain intervals,

wherein the determining the coordinates corresponding to the first weights indicating the representative values of the first sharpening kernel comprises determining the coordinates corresponding to the first weights indicating the representative values of the first sharpening kernel, based on a comparison of a result of applying the third sharpening kernel to the image with a result of applying the fourth sharpening kernel to the image.

7. The method of claim 6 , wherein a number of weights in the fourth sharpening kernel is greater than a number of weights in the third sharpening kernel.

8. The method of claim 5 , further comprising, based on one or more of the new weights being included in the first weights indicating the representative values of the first sharpening kernel, generating pixels corresponding to the one or more of the new weights by performing interpolation on the pixels in the image.

9. The method of claim 1 , wherein the determining the coordinates corresponding to the first weights indicating the representative values of the first sharpening kernel comprises:

applying a high pass filter to the first sharpening kernel; and

determining, based on absolute values of the first sharpening kernel and absolute values of the first sharpening kernel to which the high pass filter is applied, the coordinates corresponding to the first weights indicating the representative values of the first sharpening kernel.

10. The method of claim 9 , wherein the first weights in the second sharpening kernel are weights at positions symmetric with respect to a center point of the second sharpening kernel, and

a maximum value among the first weights in the second sharpening kernel is located at the center point of the second sharpening kernel.

11. An electronic device comprising:

a communication interface;

a camera;

a memory storing one or more instructions; and

at least one processor configured to execute the one or more instructions stored in the memory to:

obtain an image generated by the camera,

obtain a first sharpening kernel for enhancing sharpness of the image, wherein the first sharpening kernel is data including a plurality of weights to be applied to pixels in the image, and the data is of a lower resolution than the image,

determine coordinates corresponding to first weights indicating representative values of the first sharpening kernel from among the plurality of weights in the first sharpening kernel, wherein the representative values are values indicating feature points of a shape of the first sharpening kernel,

generate a second sharpening kernel by selecting the first weights corresponding to the determined coordinates, and

obtain a sharpened image by applying the second sharpening kernel to the image.

12. The electronic device of claim 11 , wherein the at least one processor is further configured to execute the one or more instructions to:

obtain a reference image by using the camera;

calculate a point spread function based on the reference image; and

obtain the first sharpening kernel based on the point spread function.

13. The electronic device of claim 11 , wherein the at least one processor is further configured to execute the one or more instructions to determine as zero point coordinates of a weight, from among the plurality of weights, corresponding to a center point of the first sharpening kernel.

14. The electronic device of claim 11 , wherein the at least one processor is further configured to execute the one or more instructions to obtain an augmented first sharpening kernel including new weights by performing interpolation between adjacent weights among the plurality of weights in the first sharpening kernel.

15. The electronic device of claim 14 , wherein the at least one processor is further configured to execute the one or more instructions to determine the coordinates corresponding to the first weights from among the new weights in the augmented first sharpening kernel.

16. The electronic device of claim 15 , wherein the at least one processor is further configured to execute the one or more instructions to:

obtain a third sharpening kernel by selecting second weights in the augmented first sharpening kernel;

obtain a fourth sharpening kernel by selecting third weights from the augmented first sharpening kernel at certain intervals; and

determine the coordinates corresponding to the first weights indicating the representative values of the first sharpening kernel, based on a comparison of a result of applying the third sharpening kernel to the image with a result of applying the fourth sharpening kernel to the image.

17. The electronic device of claim 15 , wherein the at least one processor is further configured to execute the one or more instructions to, based on one or more of the new weights being included in the first weights indicating the representative values of the first sharpening kernel, generate pixels corresponding to the one or more of the new weights by performing interpolation on the pixels in the image.

18. The electronic device of claim 11 , wherein the at least one processor is further configured to execute the one or more instructions to:

apply a high pass filter to the first sharpening kernel; and

determine, based on absolute values of the first sharpening kernel and absolute values of the first sharpening kernel to which the high pass filter is applied, the coordinates corresponding to the first weights indicating the representative values of the first sharpening kernel.

19. The electronic device of claim 18 , wherein the first weights in the second sharpening kernel are weights at positions symmetric with respect to a center point of the second sharpening kernel, and

a maximum value among the first weights in the second sharpening kernel is located at the center point of the second sharpening kernel.

20. A non-transitory computer-readable recording medium having recorded thereon a program that is executable by at least one process to perform the method of claim 1 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2022
From: KIES, PAWAL; CHMIELEWSKI, RADOSLAW
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 059962/0562 →
Priority Claims (1)
KR 10-2021-0119214 · Sep 7, 2021 · national
Continuity (2)
Continuation PCTKR2022006038 · Apr 27, 2022
Related Publication 20230069998A1 · Mar 9, 2023
References Cited (33)
US 8306348B2 · Cohen et al. · 2012 [cited by applicant]
US 8339481B2 · Kim et al. · 2012 [cited by applicant]
US 8588544B2 · Lei et al. · 2013 [cited by applicant]
US 8718394B2 · Vandame · 2014 [cited by applicant]
US 9165350B2 · Chang et al. · 2015 [cited by applicant]
US 9445007B1 · Li et al. · 2016 [cited by applicant]
US 9489719B2 · Naruse et al. · 2016 [cited by applicant]
US 9633417B2 · Sugimoto et al. · 2017 [cited by applicant]
US 10282822B2 · Shmunk et al. · 2019 [cited by applicant]
US 10964026B2 · Zhao et al. · 2021 [cited by applicant]
US 10991074B2 · Bousmalis et al. · 2021 [cited by applicant]
US 11030724B2 · Cho et al. · 2021 [cited by applicant]
US 20100303376A1 · Chen et al. · 2010 [cited by applicant]
US 20110311154A1 · Vandame · 2011 [cited by applicant]
US 20150078677A1 · Chang et al. · 2015 [cited by applicant]
US 20160080711A1 · Oniki · 2016 [cited by examiner]
US 20160171667A1 · Tezaur · 2016 [cited by examiner]
US 20160248984A1 · Li et al. · 2016 [cited by applicant]
US 20160335747A1 · Yang · 2016 [cited by examiner]
US 20170004603A1 · Irie · 2017 [cited by examiner]
US 20170053380A1 · McNally · 2017 [cited by examiner]
US 20170365046A1 · Tezaur · 2017 [cited by examiner]
US 20200218936A1 · Yadav et al. · 2020 [cited by applicant]
US 20210117768A1 · Liu et al. · 2021 [cited by applicant]
US 20210183015A1 · Ahn · 2021 [cited by examiner]
KR 101313911B1 · 2013 [cited by applicant]
KR 1020190089922A · 2019 [cited by applicant]
KR 1020200030988A · 2020 [cited by applicant]
KR 1020200031012A · 2020 [cited by applicant]
KR 1020200084808A · 2020 [cited by applicant]
International Search Report (PCT/ISA/220 and PCT/ISA/210) and Written Opinion (PCT/ISA/237) issued Aug. 5, 2022 by the International Searching Authority in International Application No. PCT/KR2022/006038. [cited by applicant]
“Wiener deconvolution”, Wikipedia, Jan. 3, 2022, 4 pages total, https://en.wikipedia.org/wiki/Wiener_deconvolution. [cited by applicant]
“Bilinear interpolation”, Wikipedia, Apr. 10, 2022, 4 pages total, https://en.wikipedia.org/wiki/Bilinear_interpolation. [cited by applicant]