Granted Patent B2

US 12,456,172 · App. 17/972,961 · Granted Oct 28, 2025

Inventors: Zengli Yang (San Jose, CA); Long Bao (San Diego, CA); Shuangquan Wang (San Diego, CA); Dongwoon Bai (San Diego, CA); Jungwon Lee (San Diego, CA)

Assignee: Samsung Electronics Co., Ltd.

G06T5/70G06N3/045G06N3/084G06T5/30G06T5/50G06T2207/20016G06T2207/20081G06T2207/20084G06T2207/20224

Abstract

Claims (74)

1. A method comprising:

computing noise data by subtracting, by a processing circuit, a noisy image from a corresponding ground truth image;

clustering, by the processing circuit, a plurality of noise values of the noise data based on intensity values of the corresponding ground truth image;

permuting, by the processing circuit, a plurality of locations of the noise values of the noise data within each cluster;

generating, by the processing circuit, a synthetic noise image based on the permuted locations of the noise values;

adding, by the processing circuit, the synthetic noise image to the corresponding ground truth image to generate a synthetic noisy image; and

augmenting an image dataset for training a neural network to perform image denoising with the synthetic noisy image.

2. The method of claim 1 , further comprising:

permuting, by the processing circuit, the plurality of locations of the noise values of the noise data within each cluster to compute second permuted locations of the noise values;

generating, by the processing circuit, a second synthetic noise image based on the second permuted locations of the noise values;

adding, by the processing circuit, the second synthetic noise image to the corresponding ground truth image to generate a second synthetic noisy image different from the synthetic noisy image; and

augmenting the image dataset with the second synthetic noisy image.

3. The method of claim 1 , wherein the noisy image is one of a plurality of noisy images associated with the corresponding ground truth image; and

wherein the corresponding ground truth image is generated by fusing the plurality of noisy images to generate a synthetic noise-free image.

4. The method of claim 3 , further comprising:

generating a first plurality of synthetic noisy images each based on a pairing of one of the plurality of noisy images and the synthetic noise-free image; and

augmenting the image dataset with the plurality of synthetic noisy images.

5. The method of claim 4 , further comprising:

generating a second plurality of synthetic noisy images based on the pairing of one of the plurality of noisy images and the synthetic noise-free image, each of the second plurality of synthetic noisy images corresponding to a different permutation of the locations of the noise values; and

augmenting the image dataset with the second plurality of synthetic noisy images.

6. The method of claim 4 , further comprising:

generating a synthetic noisy image for each pairing of the plurality of noisy images and the corresponding ground truth images; and

augmenting the image dataset with the synthetic noisy image for each pairing of the plurality of noisy images and the corresponding ground truth images.

7. The method of claim 5 , further comprising generating a plurality of synthetic noisy images for each pairing of the plurality of noisy images and the corresponding ground truth images, each of the plurality of synthetic noisy images corresponding to a different permutation of the locations of the noise values.

8. The method of claim 1 , further comprising training the neural network to perform image denoising, the training being performed using the image dataset augmented by the synthetic noisy image,

wherein the neural network comprises a multi-scale residual dense block (MRDB), the MRDB comprising:

a residual dense block (RDB) comprising a plurality of convolutional modules; and

an atrous spatial pyramid pooling (ASPP) module.

9. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to:

compute noise data by subtracting a noisy image from a corresponding ground truth image;

cluster a plurality of noise values of the noise data based on intensity values of the corresponding ground truth image;

permute a plurality of locations of the noise values of the noise data within each cluster;

generate a synthetic noise image based on the permuted locations of the noise values;

add the synthetic noise image to the corresponding ground truth image to generate a synthetic noisy image; and

augment an image dataset for training a neural network to perform image denoising with the synthetic noisy image.

10. The non-transitory computer-readable medium of claim 9 , further storing instructions that, when executed by the processor, cause the processor to:

permute the plurality of locations of the noise values of the noise data within each cluster to compute second permuted locations of the noise values;

generate a second synthetic noise image based on the second permuted locations of the noise values;

add the second synthetic noise image to the corresponding ground truth image to generate a second synthetic noisy image different from the synthetic noisy image; and

augment the image dataset with the second synthetic noisy image.

11. The non-transitory computer-readable medium of claim 9 , wherein the noisy image is one of a plurality of noisy images associated with the corresponding ground truth image; and

wherein the corresponding ground truth image is generated by fusing the plurality of noisy images to generate a synthetic noise-free image.

12. The non-transitory computer-readable medium of claim 11 , further storing instructions that, when executed by the processor, cause the processor to:

generate a first plurality of synthetic noisy images each based on a pairing of one of the plurality of noisy images and the synthetic noise-free image; and

augment the image dataset with the plurality of synthetic noisy images.

13. The non-transitory computer-readable medium of claim 12 , further storing instructions that, when executed by the processor, cause the processor to:

generate a second plurality of synthetic noisy images based on the pairing of one of the plurality of noisy images and the synthetic noise-free image, each of the second plurality of synthetic noisy images corresponding to a different permutation of the locations of the noise values; and

augment the image dataset with the second plurality of synthetic noisy images.

14. The non-transitory computer-readable medium of claim 12 , further storing instructions that, when executed by the processor, cause the processor to:

generate a synthetic noisy image for each pairing of the plurality of noisy images and the corresponding ground truth images; and

augment the image dataset with the synthetic noisy image for each pairing of the plurality of noisy images and the corresponding ground truth images.

15. The non-transitory computer-readable medium of claim 14 , further storing instructions that, when executed by the processor, cause the processor to generate a plurality of synthetic noisy images for each pairing of the plurality of noisy images and the corresponding ground truth images, each of the plurality of synthetic noisy images corresponding to a different permutation of the locations of the noise values.

16. A system comprising:

a processing circuit; and

a memory storing instructions that, when executed by the processing circuit, cause the processing circuit to:

compute noise data by subtracting a noisy image from a corresponding ground truth image;

cluster a plurality of noise values of the noise data based on intensity values of the corresponding ground truth image;

permute a plurality of locations of the noise values of the noise data within each cluster;

generate a synthetic noise image based on the permuted locations of the noise values;

add the synthetic noise image to the corresponding ground truth image to generate a synthetic noisy image; and

augment an image dataset for training a neural network to perform image denoising with the synthetic noisy image.

17. The system of claim 16 , wherein the memory further stores instructions that, when executed by the processing circuit, cause the processing circuit to:

permute the plurality of locations of the noise values of the noise data within each cluster to compute second permuted locations of the noise values;

generate a second synthetic noise image based on the second permuted locations of the noise values;

add the second synthetic noise image to the corresponding ground truth image to generate a second synthetic noisy image different from the synthetic noisy image; and

augment the image dataset with the second synthetic noisy image.

18. The system of claim 16 , wherein the noisy image is one of a plurality of noisy images associated with the corresponding ground truth image; and

wherein the corresponding ground truth image is generated by fusing the plurality of noisy images to generate a synthetic noise-free image.

19. The system of claim 18 , wherein the memory further stores instructions that, when executed by the processing circuit, cause the processing circuit to:

generate a first plurality of synthetic noisy images each based on a pairing of one of the plurality of noisy images and the synthetic noise-free image; and

augment the image dataset with the plurality of synthetic noisy images.

20. The system of claim 19 , wherein the memory further stores instructions that, when executed by the processing circuit, cause the processing circuit to:

generate a second plurality of synthetic noisy images based on the pairing of one of the plurality of noisy images and the synthetic noise-free image, each of the second plurality of synthetic noise images corresponding to a different permutation of the locations of the noise values; and

augment the image dataset with the second plurality of synthetic noisy images.

Assignments (1)

Continuity (4)

References Cited (85)