Method and apparatus for joint image processing and perception

1. A method of machine learning comprising:

acquiring a plurality of raw images;

employing at least one hardware processor to execute processes of:

determining a representation of a raw image of said plurality of raw images, comprising:

variance-stabilizing pixel-value transformation of said raw image;

cascaded activation of:

soft camera projection; and

image projection;

and

inverse pixel-value transformation;

initializing a plurality of representation parameters of said representation;

defining a plurality of analysis parameters of an image analysis network configured to process said representation; and

jointly training said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function;

thereby producing a learned machine.

2. The method of claim 1 wherein said pixel-value transformation is an Anscombe transformation and said inverse pixel-value transformation is an unbiased inverse Anscombe transformation.

3. The method of claim 2 further comprising generating an added channel.

4. The method of claim 1 wherein said image projection comprises performing steps of multi-level spatial convolution, pooling, subsampling, and interpolation.

5. The method of claim 4 wherein said plurality of representation parameters comprises a number of levels for said multi-level spatial convolution, a pooling parameter, a stride of said subsampling, and a step of said interpolation.

6. The method of claim 4 further comprising:

evaluating said learned machine using a plurality of test images; and

revising said multi-level spatial convolution, said pooling, said subsampling, and said interpolation according to a result of said evaluating.

7. A method of machine learning comprising:

acquiring a plurality of raw images;

employing at least one hardware processor to execute processes of:

determining a representation of a raw image of said plurality of raw images;

initializing a plurality of representation parameters of said representation;

defining a plurality of analysis parameters of an image analysis network configured to process said representation; and

jointly training said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, comprising formulating said combined objective function as a nested bilevel objective function comprising an outer objective function relevant to said image analysis network and an inner objective function relevant to said representation;

thereby producing a learned machine.

8. The method of claim 7 , further comprising:

updating said plurality of raw images and evaluating said learned machine using an updated plurality of raw images; and

revising said plurality of representation parameters based on results of said evaluating.

9. A method of machine learning comprising:

acquiring a plurality of raw images;

employing at least one hardware processor to execute processes of:

determining a representation of a raw image of said plurality of raw images;

initializing a plurality of representation parameters of said representation;

defining a plurality of analysis parameters of an image analysis network configured to process said representation;

jointly training said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, thereby producing a learned machine;

evaluating said learned machine using a plurality of test images;

adding selected test images to said plurality of raw images; and

repeating said determining, initializing, defining, and jointly training;

thereby continually updating said plurality of representation parameters and said plurality of analysis parameters.

10. A method of machine learning comprising:

acquiring a plurality of raw images;

employing at least one hardware processor to execute processes of:

determining a representation of a raw image of said plurality of raw images;

initializing a plurality of representation parameters of said representation;

defining a plurality of analysis parameters of an image analysis network configured to process said representation;

jointly training said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, thereby producing a learned machine; and

cyclically operating said learned machine in alternate modes:

during a first mode:

updating said plurality of raw images; and

executing said processes of determining, initializing, defining, and jointly training;

and

during a second mode, classifying new images according to latest values of said plurality of representation parameters and said plurality of analysis parameters.

11. The method of claim 10 , further comprising:

evaluating said learned machine using an updated plurality of raw images; and

revising said plurality of representation parameters based on results of said evaluating.

12. A learning machine comprising:

an image acquisition device for acquiring a plurality of raw images;

a memory device having processor executable instructions stored thereon for execution by a hardware processor, causing the hardware processor to:

determine a representation of a raw image of said plurality of raw images, comprising:

transforming pixel-values of said raw image to produce a transformed image comprising pixels of variance-stabilized values;

successively performing processes of:

soft camera projection; and

image projection;

and

inverse transforming said transformed pixels;

initialize a plurality of representation parameters of said representation;

define a plurality of analysis parameters of an image analysis network configured to process said representation; and

jointly train said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function.

13. The learning machine of claim 12 wherein said processor executable instructions further cause the hardware processor to:

implement an Anscombe transformation for said transforming pixel-values; and

implement an unbiased inverse Anscombe transformation for said inverse transforming.

14. The learning machine of claim 13 wherein said processor executable instructions further cause said hardware processor to generate an additional channel to said transformed image.

15. The learning machine of claim 12 wherein said processor executable instructions causing said image projection comprise computer executable instructions causing said hardware processor to perform multi-level spatial convolution, pooling, subsampling, and interpolation.

16. The learning machine of claim 15 wherein said said plurality of representation parameters comprises a number of levels for said multi-level spatial convolution, a pooling parameter, a stride of said subsampling, and a step of said interpolation.

17. The learning machine of claim 15 wherein said processor executable instructions further cause said hardware processor to perform processes of:

performance evaluation using a plurality of test images; and

revising said multi-level spatial convolution, said pooling, said subsampling, and said interpolation according to a result of said performance evaluation.

18. A learning machine comprising:

an image acquisition device for acquiring a plurality of raw images;

a memory device having processor executable instructions stored thereon for execution by a hardware processor, causing the hardware processor to:

determine a representation of a raw image of said plurality of raw images;

initialize a plurality of representation parameters of said representation;

define a plurality of analysis parameters of an image analysis network configured to process said representation; and

jointly train said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, comprising joint optimization of nested bilevel objective functions, thereby enabling formulation of said combined objective function as an outer objective function relevant to said image analysis network and an inner objective function relevant to said representation.

19. A learning machine comprising:

an image acquisition device for acquiring a plurality of raw images;

a memory device having processor executable instructions stored thereon for execution by a hardware processor, causing the hardware processor to:

determine a representation of a raw image of said plurality of raw images;

initialize a plurality of representation parameters of said representation;

define a plurality of analysis parameters of an image analysis network configured to process said representation;

jointly train said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function, thereby producing the learning machine;

evaluate performance of the learning machine using a plurality of test images;

add selected test images to said plurality of raw images; and

repeat said determine, initialize, define and jointly train, thereby enabling continual training of the learning machine.

20. A learning machine comprising:

an image acquisition device for acquiring a plurality of raw images;

a memory device having processor executable instructions stored thereon for execution by a hardware processor, causing the hardware processor to:

determine a representation of a raw image of said plurality of raw images;

initialize a plurality of representation parameters of said representation;

define a plurality of analysis parameters of an image analysis network configured to process said representation;

jointly train said plurality of representation parameters and said plurality of analysis parameters to optimize a combined objective function; and

perform a cyclic bimodal operation wherein:

during a first mode:

said plurality of raw images is updated; and

said processes of determining, initializing, defining, and jointly training are executed;

and

during a second mode, new images are classified according to latest values of said plurality of representation parameters and said plurality of analysis-parameters.