System and method for end-to-end-differentiable joint image refinement and perception

1. A system for end-to-end differentiable joint image refinement and perception, comprising:

a processor;

a learning machine, having a memory having computer readable instructions stored thereon for execution by the processor, causing the processor to:

determine a representation of a raw image of said plurality of raw images;

initialize a plurality of representation parameters of said representation;

define a plurality of analysis parameters of an image analysis network configured to process said representation;

jointly train said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function; and

perform a cyclic bimodal operation wherein:

during a first mode:

said plurality of raw images is updated; and

said processes of determining, initializing, defining, and jointly training are executed;

 and

during a second mode, new images are classified according to latest values of said plurality of representation parameters and said plurality of analysis parameters.

2. The system of claim 1 , further comprising an image acquisition module for acquiring the plurality of raw images.

3. The system of claim 1 , wherein the computer readable instructions further cause the processor to:

update the plurality of raw images and evaluate the learned machine using an updated plurality of raw images; and

revise the plurality of representation parameters based on results of evaluation.

4. The system of claim 1 , further comprising a learning depot comprising training data and learned data.

5. The system of claim 1 , wherein the image refinement comprises at least one of the following:

demosaicing;

denoising;

deblurring;

tone mapping.

6. The system of claim 1 , wherein the perception comprises image classification.

7. The system of claim 1 , wherein the processor readable instructions further cause the processor to:

implement an Anscombe transformation for the transforming pixel-values; and

implement an unbiased inverse Anscombe transformation for the inverse transforming.

8. The system of claim 7 , wherein the processor readable instructions further cause the processor to generate an additional channel to the transformed image.

9. The system of claim 7 , wherein the processor readable instructions causing the image projection comprise computer readable instructions to perform multi-level spatial convolution, pooling, subsampling, and interpolation.

10. The system of claim 9 , wherein the plurality of representation parameters comprises a number of levels for the multi-level spatial convolution, a pooling parameter, a stride of the subsampling, and a step of the interpolation.

11. The system of claim 9 , wherein the processor readable instructions further cause the processor to:

evaluate a performance using a plurality of test images; and

revise the multi-level spatial convolution, the pooling, the subsampling, and the interpolation according to a result of evaluating the performance.

12. A learning machine for joint image refinement and perception, comprising:

a memory device having processor executable instructions stored thereon for execution by a hardware processor, forming:

means for determining a representation of a raw image of said plurality of raw images;

means for initializing a plurality of representation parameters of said representation;

means for defining a plurality of analysis parameters of an image analysis network configured to process said representation;

means for jointly training said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function; and

means for performing a cyclic bimodal operation wherein:

during a first mode:

said plurality of raw images is updated; and

said processes of determining, initializing, defining, and jointly training are executed;

 and

during a second mode, new images are classified according to latest values of said plurality of representation parameters and said plurality of analysis parameters.

13. The learning machine of claim 12 , further comprising:

means for updating the plurality of raw images and evaluating a performance of the learned machine using an updated plurality of raw images; and

means for revising the plurality of representation parameters based on the evaluating.

14. The learning machine of claim 12 , wherein the image refinement comprises at least one of the following:

demosaicing;

denoising;

deblurring;

tone mapping.

15. The learning machine of claim 12 , wherein the perception comprises image classification.

16. The learning machine of claim 12 , wherein the means for determining further comprise:

means for performing an Anscombe transformation for transforming pixel-values of the raw image; and

means for performing an unbiased inverse Anscombe transformation.

17. The learning machine of claim 16 , further comprising means for generating an additional channel to the transformed image.

18. The learning machine of claim 16 , wherein the image projection comprises multi-level spatial convolution, pooling, subsampling, and interpolation.

19. The learning machine of claim 18 , wherein the plurality of representation parameters comprises a number of levels for the multi-level spatial convolution, a pooling parameter, a stride of the subsampling, and a step of the interpolation.

20. The learning machine of claim 18 , further comprising:

means for evaluating a performance of the learning machine using a plurality of test images; and

means for revising the multi-level spatial convolution, the pooling, the subsampling, and the interpolation based on the evaluating.

21. A method of machine learning, comprising:

employing a hardware processor for joint image refinement and perception, comprising:

determining a representation of a raw image of said plurality of raw images;

initializing a plurality of representation parameters of said representation;

defining a plurality of analysis parameters of an image analysis network configured to process said representation;

jointly training said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, thereby producing a learned machine; and

cyclically operating said learned machine in alternate modes:

during a first mode:

updating said plurality of raw images; and

executing said processes of determining, initializing, defining, and jointly training; and

during a second mode, classifying new images according to latest values of said plurality of representation parameters and said plurality of analysis parameters.