Apparatus and method for white balance editing

1. An apparatus for white balance editing, the apparatus comprising:

a memory storing instructions; and

at least one processor configured to execute the instructions to:

obtain an input image having a corrected white balance that is obtained by correcting an original white balance via image signal processing;

obtain, using a first neural network, an intermediate representation of the obtained input image, the intermediate representation having the original white balance that is not corrected by the image signal processing; and

obtain, using a second neural network, a first output image having a first white balance different than the original white balance, based on the obtained intermediate representation.

2. The apparatus of claim 1 , wherein the at least one processor is further configured to execute the instructions to obtain, using a third neural network, a second output image having a second white balance different than the original white balance and the first white balance, based on the obtained intermediate representation.

3. The apparatus of claim 2 , wherein the at least one processor is further configured to execute the instructions to:

control to display a slider for selecting one among a plurality of white balances; and

based on a user input of selecting the one among the plurality of white balances, using the displayed slider, control to display a result image having the selected one among the plurality of white balances, using any one or any combination of the obtained first output image, the obtained second output image and a blended image of the obtained first output image and the obtained second output image.

4. The apparatus of claim 1 , wherein the at least one processor is further configured to execute the instructions to:

downsample the obtained input image;

obtain, using the first neural network, a downsampled intermediate representation of the downsampled input image, the downsampled intermediate representation having the original white balance not corrected by the image signal processing;

obtain, using the second neural network, a downsampled output image having the first white balance different than the original white balance, based on the obtained downsampled intermediate representation; and

apply a color mapping to the obtained downsampled output image to obtain the first output image having the first white balance different than the original white balance.

5. The apparatus of claim 4 , wherein the color mapping is a polynomial function based on a mapping matrix that maps colors of the downsampled input image to colors of the downsampled output image.

6. The apparatus of claim 1 , wherein the first neural network comprises:

one or more first convolutional layers, one or more first rectified linear unit (ReLU) layers and at least one first max-pooling layer; and

one or more second convolutional layers, one or more second ReLU layers and at least one second max-pooling layer, and

a second number of channels of each of the one or more second convolutional layers, the one or more second ReLU layers and the at least one second max-pooling layer is doubled a first number of channels of each of the one or more first convolutional layers, the one or more first ReLU layers and the at least one first max-pooling layer.

7. The apparatus of claim 1 , wherein the apparatus, the first neural network and the second neural network are implemented in a server, and

the at least one processor is further configured to execute the instructions to:

receive, from an image signal processor of an electronic device, the input image having the corrected white balance; and

transmit, to the electronic device, the obtained first output image.

8. A method of white balance editing, the method being performed by at least one processor and comprising:

obtaining an input image having a corrected white balance that is obtained by correcting an original white balance via image signal processing;

obtaining, using a first neural network, an intermediate representation of the obtained input image, the intermediate representation having the original white balance that is not corrected by the image signal processing; and

obtaining, using a second neural network, a first output image having a first white balance different than the original white balance, based on the obtained intermediate representation.

9. The method of claim 8 , further comprising obtaining, using a third neural network, a second output image having a second white balance different than the original white balance and the first white balance, based on the obtained intermediate representation.

10. The method of claim 9 , further comprising:

displaying a slider for selecting one among a plurality of white balances; and

based on a user input of selecting the one among the plurality of white balances, using the displayed slider, displaying a result image having the selected one among the plurality of white balances, using any one or any combination of the obtained first output image, the obtained second output image and a blended image of the obtained first output image and the obtained second output image.

11. The method of claim 8 , further comprising:

downsampling the obtained input image;

obtaining, using the first neural network, a downsampled intermediate representation of the downsampled input image, the downsampled intermediate representation having the original white balance not corrected by the image signal processing;

obtaining, using the second neural network, a downsampled output image having the first white balance different than the original white balance, based on the obtained downsampled intermediate representation; and

applying a color mapping to the obtained downsampled output image to obtain the first output image having the first white balance different than the original white balance.

12. The method of claim 11 , wherein the color mapping is a polynomial function based on a mapping matrix that maps colors of the downsampled input image to colors of the downsampled output image.

13. The method of claim 8 , wherein the first neural network comprises:

one or more first convolutional layers, one or more first rectified linear unit (ReLU) layers and at least one first max-pooling layer; and

one or more second convolutional layers, one or more second ReLU layers and at least one second max-pooling layer, and

a second number of channels of each of the one or more second convolutional layers, the one or more second ReLU layers and the at least one second max-pooling layer is doubled a first number of channels of each of the one or more first convolutional layers, the one or more first ReLU layers and the at least one first max-pooling layer.

14. The method of claim 8 , wherein the at least one processor, the first neural network and the second neural network are implemented in a server, and

the method further comprises:

receiving, from an image signal processor of an electronic device, the input image having the corrected white balance; and

transmitting, to the electronic device, the obtained first output image.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor for white balance editing, cause the at least one processor to:

obtain an input image having a corrected white balance that is obtained by correcting an original white balance via image signal processing;

obtain, using a first neural network, an intermediate representation of the obtained input image, the intermediate representation having the original white balance that is not corrected by the image signal processing; and

obtain, using a second neural network, a first output image having a first white balance different than the original white balance, based on the obtained intermediate representation.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions further cause the at least one processor to obtain, using a third neural network, a second output image having a second white balance different than the original white balance and the first white balance, based on the obtained intermediate representation.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions further cause the at least one processor to:

control to display a slider for selecting one among a plurality of white balances; and

based on a user input of selecting the one among the plurality of white balances, using the displayed slider, control to display a result image having the selected one among the plurality of white balances, using any one or any combination of the obtained first output image, the obtained second output image and a blended image of the obtained first output image and the obtained second output image.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions further cause the at least one processor to:

downsample the obtained input image;

obtain, using the first neural network, a downsampled intermediate representation of the downsampled input image, the downsampled intermediate representation having the original white balance not corrected by the image signal processing;

obtain, using the second neural network, a downsampled output image having the first white balance different than the original white balance, based on the obtained downsampled intermediate representation; and

apply a color mapping to the obtained downsampled output image to obtain the first output image having the first white balance different than the original white balance.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the color mapping is a polynomial function based on a mapping matrix that maps colors of the downsampled input image to colors of the downsampled output image.

20. The non-transitory computer-readable storage medium of claim 15 , wherein the first neural network comprises:

one or more first convolutional layers, one or more first rectified linear unit (ReLU) layers and at least one first max-pooling layer; and

one or more second convolutional layers, one or more second ReLU layers and at least one second max-pooling layer, and

a second number of channels of each of the one or more second convolutional layers, the one or more second ReLU layers and the at least one second max-pooling layer is doubled a first number of channels of each of the one or more first convolutional layers, the one or more first ReLU layers and the at least one first max-pooling layer.