IP Library Granted Patent US 11,792,535
Granted Patent B2
US 11,792,535 · App. 17/586,490 · Granted Oct 17, 2023

System and method to improve quality in under-display camera system with radially increasing distortion

Inventors: Yibo Xu (Plano, TX); Hamid R Sheikh (Allen, TX)
Assignee: Samsung Electronics Co., Ltd.
H04N23/80H04N23/76H04N23/84G06F1/1686
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,792,535
App. No.
17/586,490
Granted
Oct 17, 2023
Kind
B2
Abstract

A system and method are provided to improve quality in an under-display camera (UDC) system with radially-increasing distortion. At least one processor receives an image and performs multi-frame processing, image signal processing, and a point spread function inversion (PSFI) on the image to produce a processed image. A PSFI radial coring is applied on the processed image to reduce noise in the processed image resulting from the PSFI. The at least one processor can apply a chroma suppression to the processing of the image to reduce brightness and color saturation is select areas in the processed image. Image restoration can be performed on the processed image to produce an output image. The image restoration may include generating a dither signal, applying a dither signal corner attenuation to the dither signal, combining the attenuated dither signal with a sharpened denoised signal, and applying a halo suppression on the combined signals.

Claims (51)

1. An electronic device comprising:

a display;

an imaging system; and

at least one processor configured to:

obtain an image captured by the imaging system, the image comprising one or more frames;

perform multi-frame processing and image signal processing on the image to produce a first processed image;

perform a point spread function inversion (PSFI) on the first processed image to produce a second processed image; and

apply PSFI radial coring on the second processed image, wherein the PSFI radial coring is configured to reduce noise in the second processed image resulting from the PSFI.

2. The electronic device of claim 1 , wherein, to apply the PSFI radial coring, the at least one processor is configured to:

apply a PSFI correction signal by subtracting the first processed image from the second processed image;

obtain an attenuation radial map from a radial map and a radially-adaptive gain curve;

modify the PSFI correction signal by element-wise multiplication with the attenuation radial map; and

add the modified PSFI correction signal back to the first processed image to produce a third processed image.

3. The electronic device of claim 2 , wherein the attenuation radial map is configured to apply a gain parameter to attenuate the PSFI correction signal on a pixel basis.

4. The electronic device of claim 3 , wherein the gain parameter is based on a radial adaptive gain function configured to apply a gain value as a function of a threshold value and slope.

5. The electronic device of claim 2 , wherein the attenuation radial map is configured to attenuate the PSFI correction signal more towards corners of the second processed image.

6. The electronic device of claim 1 , wherein the at least one processor is configured to apply the PSFI radial coring in one of: a YUV domain or an RGB domain.

7. The electronic device of claim 1 , wherein the imaging system comprises one of:

a single camera;

a multiple camera system; or

an under display camera (UDC).

8. A method comprising:

obtaining an image, the image comprising one or more frames;

performing multi-frame processing and image signal processing on the image to produce a first processed image;

performing a point spread function inversion (PSFI) on the first processed image to produce a second processed image; and

applying PSFI radial coring on the second processed image, wherein the PSFI radial coring is configured to reduce noise in the second processed image resulting from the PSFI.

9. The method of claim 8 , wherein applying the PSFI radial coring comprises:

applying a PSFI correction signal by subtracting the first processed image from the second processed image;

obtaining an attenuation radial map from a radial map and a radially-adaptive gain curve;

modifying the PSFI correction signal by element-wise multiplication with the attenuation radial map; and

adding the modified PSFI correction signal back to the first processed image to produce a third processed image.

10. The method of claim 9 , wherein the attenuation radial map is configured to apply a gain parameter to attenuate the PSFI correction signal on a pixel basis.

11. The method of claim 10 , wherein the gain parameter is generated by a radial adaptive gain function configured to apply a gain value as a function of a threshold value and slope.

12. The method of claim 9 , wherein the attenuation radial map is configured to attenuate the PSFI correction signal more towards corners of the second processed image.

13. The method of claim 9 , further comprising:

performing image restoration on the third processed image to produce an output image.

14. The method of claim 8 , wherein applying the PSFI radial coring comprises applying the PSFI coring in one of: a YUV domain or an RGB domain.

15. A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:

obtain an image, the image comprising one or more frames;

perform multi-frame processing and image signal processing on the image to produce a first processed image;

perform a point spread function inversion (PSFI) on the first processed image to produce a second processed image; and

apply PSFI radial coring on the second processed image, wherein the PSFI radial coring is configured to reduce noise in the second processed image resulting from the PSFI.

16. The non-transitory computer readable medium of claim 15 , wherein the instructions that when executed cause the at least one processor to apply the PSFI radial coring comprise instructions that when executed cause the at least one processor to:

apply a PSFI correction signal by subtracting the first processed image from the second processed image;

obtain an attenuation radial map from a radial map and a radially-adaptive gain curve;

modify the PSFI correction signal by element-wise multiplication with the attenuation radial map; and

add the modified PSFI correction signal back to the first processed image to produce a third processed image.

17. The non-transitory computer readable medium of claim 16 , wherein the attenuation radial map is configured to apply a gain parameter to attenuate the PSFI correction signal on a pixel basis.

18. The non-transitory computer readable medium of claim 17 , wherein the gain parameter is based on a radial adaptive gain function configured to apply a gain value as a function of a threshold value and slope.

19. The non-transitory computer readable medium of claim 16 , wherein the attenuation radial map is configured to attenuate the PSFI correction signal more towards corners of the second processed image.

20. The non-transitory computer readable medium of claim 15 , wherein the instructions when executed cause the at least one processor to apply the PSFI radial coring in one of: a YUV domain or an RGB domain.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 27, 2022
From: XU, YIBO; SHEIKH, HAMID R.
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 058798/0687 →
Continuity (2)
Provisional Application 63231594 · Aug 10, 2021
Related Publication 20230053084A1 · Feb 16, 2023