Unified optimization method for end-to-end camera image processing for translating a sensor captured image to a display image

1. A computer implemented method of extracting a latent image from an observed image, said method comprising:

implementing a plurality of image processing operations within a single optimization framework, wherein said single optimization framework comprises solving a linear minimization expression;

mapping said linear minimization expression onto at least one non-linear solver; and

using said non-linear solver, iteratively solving said linear minimization expression in order to extract said latent image from said observed image, wherein said linear minimization expression comprises: a data term, and a regularization term, and wherein said regularization term comprises a plurality of non-linear image priors.

2. The method of claim 1 , wherein said data term comprises a linear least-squares expression.

3. The method of claim 1 , wherein said regularization term comprises three image priors, wherein said three image priors are selected from the group consisting of: total variation image prior, a denoising image prior, and a cross-channel gradient correlation image prior.

4. The method of claim 3 , wherein said denoising prior can be selected from a group consisting of: BM3D type, NLM type, and sliding DCT type.

5. The method of claim 1 , wherein said single optimization framework is implemented to execute on a graphics processing unit (GPU).

6. The method of claim 1 , wherein said non-linear solver is selected from a group consisting of: a primal-dual solver and an ADMM solver.

7. The method of claim 1 , wherein said single optimization framework comprises a forward image formation model, wherein said forward image formation model comprises a sequence of independent linear transformations.

8. The method of claim 7 , wherein said forward image formation model can be selected from a group consisting of: joint Bayer demosaicking and denoising, interlaced HDR reconstruction, image fusion from color camera arrays, super-resolution, and joint image stack denoising and demosaicking.

9. A non-transitory computer-readable storage medium having stored thereon, computer executable instructions that, if executed by a computer system cause the computer system to perform a method of determining a latent image from an observed image, said method comprising:

implementing a plurality of image processing operations within a single optimization framework, wherein said single optimization framework comprises solving a linear minimization expression;

mapping said linear minimization expression onto at least one non-linear solver; and

using said non-linear solver, iteratively solving said linear minimization expression in order to extract said latent image from said observed image, wherein said linear minimization expression comprises: a data term, and a regularization term, and wherein said regularization term comprises a plurality of non-linear image priors.

10. The non-transitory computer-readable medium as described in claim 9 , wherein said data term comprises a linear least-squares expression.

11. The non-transitory computer-readable medium as described in claim 9 , wherein said regularization term comprises three image priors, wherein said three image priors are selected from the group consisting of: total variation image prior, a denoising image prior, and a cross-channel gradient correlation image prior.

12. The non-transitory computer-readable medium as described in claim 11 , wherein said denoising prior can be selected from a group consisting of: BM3D type, NLM type, and sliding DCT type.

13. The non-transitory computer-readable medium as described in claim 9 , wherein said single optimization framework is implemented to execute on a graphics processing unit (GPU).

14. The non-transitory computer-readable medium as described in claim 9 , wherein said non-linear solver is selected from a group consisting of: a primal-dual solver and an ADMM solver.

15. The non-transitory computer-readable medium as described in claim 9 , wherein said single optimization framework comprises a forward image formation model, wherein said forward image formation model comprises a sequence of independent linear transformations.

16. The non-transitory computer-readable medium as described in claim 15 , wherein said forward image formation model can be selected from a group consisting of: joint Bayer demosaicking and denoising, interlaced HDR reconstruction, image fusion from color camera arrays, super-resolution, and joint image stack denoising and demosaicking.

17. A system for providing a latent image from an observed image, said system comprising:

a memory storing information related to an image construction framework;

a processor coupled to said memory, said processor operable to implement a method of providing a latent image from an observed image, said method comprising:

integrating a plurality of image processing operations within an optimization framework, wherein said optimization framework comprises solving a linear minimization equation;

mapping said linear minimization equation onto at least one non-linear solver; and

using said non-linear solver, iteratively solving said linear minimization equation in order to extract said latent image from said observed image, wherein said linear minimization equation comprises: a data term, a regularization term, and wherein said regularization term comprises a plurality of non-linear image priors.

18. The system of claim 17 , wherein said data term comprises a linear least-squares expression.

19. The system of claim 17 , wherein said regularization term comprises three image priors, wherein said three image priors are selected from the group consisting of: total variation image prior, a denoising image prior, and a cross-channel gradient correlation image prior.

20. The system of claim 17 , wherein said optimization framework is specialized to execute on a graphics processing unit (GPU).