System and method for joint image refinement and perception

1. A system for end-to-end differentiable joint image refinement and perception, comprising:

a processor;

a learning machine, having a memory having computer readable instructions stored thereon for execution by the processor, causing the processor to:

determine a representation of a raw image of a plurality of raw images;

initialize a plurality of representation parameters of said representation;

define a plurality of analysis parameters of an image analysis network configured to process said representation;

jointly train said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, thereby producing the learning machine;

evaluate performance of the learning machine using a plurality of test images;

add selected test images to said plurality of raw images; and

repeat said determine, initialize, define and jointly train, thereby continually training the learning machine.

2. The system of claim 1 , further comprising an image acquisition module for acquiring the plurality of raw images.

3. The system of claim 1 , wherein the computer readable instructions further cause the processor to:

update the plurality of raw images and evaluate the learned machine using an updated plurality of raw images; and

revise the plurality of representation parameters based on results of evaluation.

4. The system of claim 1 , further comprising a learning depot comprising training data and learned data.

5. The system of claim 1 , wherein the image refinement comprises at least one of the following:

demosaicing;

denoising;

deblurring;

tone mapping.

6. The system of claim 1 , wherein the perception comprises image classification.

7. The system of claim 1 , wherein the processor readable instructions to determine the representation of the raw image further cause the processor to:

implement an Anscombe transformation for transforming pixel-values of the raw image; and

implement an unbiased inverse Anscombe transformation for inverse transforming.

8. The system of claim 7 , wherein the processor readable instructions further cause the processor to generate an additional channel to the transformed image.

9. The system of claim 1 , wherein the processor readable instructions to determine the representation of the raw image further cause the processor to perform multi-level spatial convolution, pooling, subsampling, and interpolation.

10. The system of claim 9 , wherein the plurality of representation parameters comprises a number of levels for the multi-level spatial convolution, a pooling parameter, a stride of the subsampling, and a step of the interpolation.

11. The system of claim 9 , wherein the processor readable instructions further cause the processor to revise the multi-level spatial convolution, the pooling, the subsampling, and the interpolation according to a result of evaluating the performance.

12. A learning machine for joint image refinement and perception, comprising:

a memory having computer readable instructions stored thereon for execution by a processor, forming:

means for determining a representation of a raw image of a plurality of raw images,

means for initializing a plurality of representation parameters of said representation;

means for defining a plurality of analysis parameters of an image analysis network configured to process said representation;

means for jointly training said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, thereby producing the learning machine;

means for evaluating performance of the learning machine using a plurality of test images;

means for adding selected test images to said plurality of raw images; and

means for repeating said determining, initializing, defining and jointly training, thereby continually training the learning machine.

13. The learning machine of claim 12 , further comprising:

means for updating the plurality of raw images and evaluating a performance of the learned machine using an updated plurality of raw images; and

means for revising the plurality of representation parameters based on the evaluating.

14. The learning machine of claim 12 , wherein the image refinement comprises at least one of the following:

demosaicing;

denoising;

deblurring;

tone mapping.

15. The learning machine of claim 12 , wherein the perception comprises image classification.

16. The learning machine of claim 12 , wherein the means for the determining further comprise:

means for performing an Anscombe transformation for transforming pixel-values of the raw image; and

means for performing an unbiased inverse Anscombe transformation for inverse transforming.

17. The learning machine of claim 16 , further comprising means for generating an additional channel to the transformed image.

18. The learning machine of claim 16 , wherein the image projection comprises multi-level spatial convolution, pooling, subsampling, and interpolation.

19. The learning machine of claim 18 , wherein the plurality of representation parameters comprises a number of levels for the multi-level spatial convolution, a pooling parameter, a stride of the subsampling, and a step of the interpolation.

20. The learning machine of claim 18 , further comprising means for revising the multi-level spatial convolution, the pooling, the subsampling, and the interpolation based on the evaluating.

21. A method of machine learning comprising:

employing a hardware processor for joint image refinement and perception, comprising:

determining a representation of a raw image of said plurality of raw images;

initializing a plurality of representation parameters of said representation;

defining a plurality of analysis parameters of an image analysis network configured to process said representation;

jointly training said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, thereby producing a learning machine;

evaluating said learned machine using a plurality of test images;

adding selected test images to said plurality of raw images; and

repeating said determining, initializing, defining, and jointly training;

thereby continually training the learning machine.