Learning to estimate high-dynamic range outdoor lighting parameters

1. A computer-implemented method for training a neural network system, the method comprising:

inputting, into a neural network, a low-dynamic range training image;

determining, using the neural network, high-dynamic range lighting parameters for the low-dynamic range training image;

rendering a training scene using the high-dynamic range lighting parameters; and

adjusting the neural network based on error associated with the training scene using a loss function.

2. The computer-implemented method of claim 1 , wherein the low-dynamic range training image is a synthetic low-dynamic range image generated from a synthetic high-dynamic range image, wherein the synthetic high-dynamic range image is generated using a high-dynamic range sky image rendered with a rendered outdoor scene.

3. The computer-implemented method of claim 1 , wherein the loss function includes a rendering loss or a domain loss.

4. The computer-implemented method of claim 3 , wherein the rendering loss is determined using pixel-wise loss between the training scene and a ground-truth scene, wherein the training scene is a simple scene rendered using the high-dynamic range lighting parameters and the ground-truth scene is the simple scene rendered using known ground-truth lighting parameters.

5. The computer-implemented method of claim 3 , wherein the domain loss is determined using cross-entropy loss to determine differences between a latent vector from a real low-dynamic range image and a latent vector from a synthetic low-dynamic range image.

6. The computer-implemented method of claim 1 , wherein the training scene is rendered using pre-rendered scenes based on the determined high-dynamic lighting parameters.

7. The computer-implemented method of claim 1 , further comprising:

comparing the training scene with a ground-truth rendered scene, wherein the ground-truth rendered scene is pre-rendered using known high-dynamic range lighting parameters.

8. The computer-implemented method of claim 1 , wherein determining the high-dynamic range lighting parameters, further comprises:

estimating an intensity parameter;

estimating a cloudiness parameter;

obtaining an environmental parameter; and

combining the intensity parameter, the cloudiness parameter, and the environmental parameter to estimate the high-dynamic range lighting parameters.

9. The computer-implemented method of claim 8 , wherein the cloudiness parameter is determined using classification based on an estimated value.

10. The computer-implemented method of claim 8 , further comprising:

selecting a first pre-rendered scene using the intensity parameter and the cloudiness parameter; and

obtaining a second pre-rendered scene using the environmental parameter.

11. The computer-implemented method of claim 10 , further comprising:

combining the first pre-rendered scene and the second pre-rendered scene to generate the training scene.

12. One or more non-transitory computer-readable media having a plurality of executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform steps comprising:

receiving, via a neural network system, an input image, wherein the neural network system is trained by:

inputting a low-dynamic range training image,

determining lighting parameters for the low-dynamic range training image,

rendering a training scene using pre-rendered scenes selected using the lighting parameters, and

adjusting the neural network system based on error associated with the training scene;

generating, using the trained neural network system, high-dynamic lighting parameters for the input image; and

outputting, via the neural network system, the high-dynamic lighting parameters.

13. The media of claim 12 , further comprising:

providing the high-dynamic lighting parameters to a user device.

14. The media of claim 12 , wherein the error comprises loss determined using a rendering loss or a domain loss.

15. The media of claim 14 , wherein the rendering loss is determined using pixel-wise loss between the training scene and a ground-truth scene, wherein the training scene is a simple scene rendered using the lighting parameters and the ground-truth scene is the simple scene rendered using known ground-truth lighting parameters.

16. The media of claim 14 , wherein the domain loss is determined using cross-entropy loss to determine differences between a latent vector from a real low-dynamic range image and a latent vector from a synthetic low-dynamic range image.

17. The media of claim 16 , wherein the domain loss is weighted, wherein a domain loss weight increases from zero at a beginning of training to one at a half-way point during training of the neural network system.

18. A computing system comprising:

means for training, via a processor, a neural network system, wherein the neural network system includes a neural network trained to determine high-dynamic range lighting parameters from input training low-dynamic range images; and

means for using, via the processor, the neural network system to determine the high-dynamic range lighting parameters from an input low-dynamic range image.

19. The computing system of claim 18 , wherein the training comprises:

inputting, via the neural network system, a low-dynamic range training image;

determining, via the neural network system, lighting parameters for the low-dynamic range training image;

rendering a training scene using pre-rendered scenes selected using the lighting parameters; and

adjusting the neural network based on error associated with the training scene, wherein the error comprises a rendering loss or a domain loss.

20. The computing system of claim 19 , wherein the rendering loss is determined using pixel-wise loss between the training scene and a ground-truth scene, wherein the training scene is a simple scene rendered using the lighting parameters and the ground-truth scene is the simple scene rendered using known ground-truth lighting parameters and wherein the domain loss is determined using cross-entropy loss to determine differences between a latent vector from a real low-dynamic range image and a latent vector from a synthetic low-dynamic range image.