Apparatus and method for white balance editing
An apparatus for white balance editing, includes a memory storing instructions, and at least one processor configured to execute the instructions to obtain an input image having an original white balance that is corrected by image signal processing, and obtain, using a first neural network, an intermediate representation of the obtained input image, the intermediate representation having the original white balance that is not corrected by the image signal processing. The at least one processor is further configured to execute the instructions to obtain, using a second neural network, a first output image having a first white balance different than the original white balance, based on the obtained intermediate representation.
1. A mobile device comprising:
a camera module including a lens module and an image sensing module configured to obtain first image data from light collected through the lens module;
a sensor module configured to obtain information about a shooting condition;
a storage module storing information about a plurality of artificial intelligence (AI) models for white-balance control;
an image signal processor configured to obtain second image data by processing at least one of the stored AI models; and
at least one application processor,
wherein the at least one application processor is configured to:
determine an AI model corresponding to the shooting condition among the plurality of AI models, for white-balance control, stored in the storage module, based on the information about the shooting condition o7btained by the sensor module; and
control the determined AI model to be loaded into the image signal processor, and
the image signal processor is further configured to input the first image data to the determined AI model so as to obtain the second image data with a white balance controlled to correspond to the shooting condition,
wherein the shooting condition is a condition for identifying whether the mobile device is located indoors or outdoors,
wherein the plurality of AI models shares a common encoder model and comprises a plurality of different decoder models.
2. The mobile device of claim 1 , wherein the first image data comprises original color filter array data.
3. The mobile device of claim 1 , wherein the first image data comprises original RGB data.
4. The mobile device of claim 1 , wherein the sensor module comprises:
a global positioning system (GPS) sensing module configured to obtain GPS sensor information;
an illumination sensing module configured to obtain illuminance sensor information; and
a network status information obtaining module configured to obtain network status information,
wherein the at least one application processor identifies whether the mobile device is located indoors or outdoors, based on at least one of the GPS sensor information, the illuminance sensor information, or the network status information, and
the AI model corresponding to a result of the identification among the plurality of AI models, for white balance control, stored in the storage module, is determined.
5. The mobile device of claim 4 , wherein, when whether the mobile device is located indoors or outdoors is determined, based on at least one of the GPS sensor information, the illumination sensor information, or the network status information, the mobile device is identified as being located indoors when the GPS sensor information is not updated and identified as being located outdoors when the GPS sensor information is updated.
6. The mobile device of claim 4 , wherein, when whether the mobile device is located indoors or outdoors is determined, based on at least one of the GPS sensor information, the illumination sensor information, or the network status information, the mobile device is identified as being located outdoors when a sensor value of the illumination sensor information is greater than or equal to a first value, and
the mobile device is identified as being located indoors when the sensor value of the illuminance sensor information is less than or equal to a second value.
7. The mobile device of claim 4 , wherein the network status information comprises at least one of received signal strength indication (RSSI) information or service set identifier (SSID) information of at least one scanned access point (AP),
wherein, when whether the mobile device is located indoors or outdoors is determined, based on at least one of the GPS sensor information, the illuminance sensor information, or the network status information, the at least one application processor is further configured to:
identify that the mobile device is located indoors when an average value of RSSIs of the scanned APs is greater than or equal to a first value and is located outdoors when the average value of the RSSIs of the scanned APs is less than a second value; and
identify that the mobile device is located indoors when the number of APs with an RSSI greater than or equal to the first value among the scanned APs is greater than or equal to a first number, and that the mobile device is located outdoors when the number of APs with an RSSI less than or equal to the second value is greater than or equal to a second number or when the number of APs, which is greater than or equal to the first value, is less than the first number.
8. A method of white balancing by a mobile device, the method comprising:
obtaining first image data;
obtaining information about a shooting condition of the mobile device;
determining an artificial intelligence (AI) model corresponding to the shooting condition among a plurality of AI models, for white-balance control, stored in a storage module of the mobile device, based on the information about the shooting condition;
controlling the determined AI model to be loaded into an image signal processor, and
inputting the first image data to the determined AI model so as to obtain a second image data with a white balance controlled to correspond to the shooting condition,
wherein the shooting condition is a condition for identifying whether the mobile device is located indoors or outdoors,
wherein the plurality of AI models shares a common encoder model and comprises a plurality of different decoder models.
9. The method of claim 8 , wherein the first image data comprises original color filter array data.
10. The method of claim 8 , wherein the first image data comprises original RGB data.
11. The method of claim 8 , wherein the plurality of AI models each comprise an encoder model and a decoder model, and comprise different decoder models.
12. The method of claim 8 , where in the shooting condition of the mobile device comprises a GPS sensor information, an illuminance sensor information and a network status information.
13. The method of claim 12 , further comprising:
identifying whether the mobile device is located indoors or outdoors, based on at least one of the GPS sensor information, the illuminance sensor information, or the network status information, and
determining the AI model corresponding to a result of the identification among the plurality of AI models, for white balance control, stored in the storage module.
14. The method of claim 13 , further comprising:
identifying the mobile device as being located indoors when the GPS sensor information is not updated; and
identifying the mobile device as being located outdoors when the GPS sensor information is updated.
15. The method of claim 13 , further comprising:
identifying the mobile device as being located outdoors when a sensor value of the illumination sensor information is greater than or equal to a first value, and
identifying the mobile device as being located indoors when the sensor value of the illuminance sensor information is less than or equal to a second value.
16. The method of claim 12 , wherein the network status information comprises at least one of received signal strength indication (RSSI) information or service set identifier (SSID) information of at least one scanned access point (AP).
17. The method of claim 16 , further comprising:
identifying that the mobile device is located indoors when an average value of at least one of RSSI of the at least one scanned AP is greater than or equal to a first value and is located outdoors when the average value of the at least one of RSSI of the at least one of scanned AP is less than a second value; and
identifying that the mobile device is located indoors when the number of AP with an RSSI greater than or equal to the first value among the at least one of scanned AP is greater than or equal to a first number, and that the mobile device is located outdoors when the number of AP with an RSSI less than or equal to the second value is greater than or equal to a second number or when the number of AP, which is greater than or equal to the first value, is less than the first number.
18. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor for white balancing cause the at least one processor to:
obtaining first image data;
obtaining information about a shooting condition of a mobile device;
determining an artificial intelligence (AI) model corresponding to the shooting condition among a plurality of AI models, for white-balance control, stored in a storage module of the mobile device, based on the information about the shooting condition obtained by a sensor module of the mobile device;
controlling the determined AI model to be loaded into an image signal processor, and
inputting the first image data to the determined AI model so as to obtain a second image data with a white balance controlled to correspond to the shooting condition,
wherein the shooting condition is a condition for identifying whether the mobile device is located indoors or outdoors,
wherein the plurality of AI models shares a common encoder model and comprises a plurality of different decoder models.