System and method for robust multi-frame demosaicing and color super resolution

1. A method of creating a super-resolved color image from a plurality of lower-resolution color images, the method comprising:

a. enforcing by a computer similarities between raw data and a high-resolution image estimate, wherein said enforcing uses a data fidelity penalty term, wherein said data fidelity penalty term is an L1 norm penalty term;

b. encouraging by a computer sharp edges in a luminance component to said high-resolution image, wherein said encouraging uses a spatial luminance penalty term;

c. encouraging by a computer smoothness in a chrominance component of said high-resolution image, wherein said encouraging uses a spatial chrominance penalty term;

d. encouraging by a computer homogeneity of an edge location and orientation in different color bands, wherein said encouraging uses an inter-color dependencies penalty term; and

e. a super-resolved color image is displayed on an computer monitor.

2. The method according to claim 1 , wherein said data fidelity penalty term is applied to space invariant point spread function, translational, affine, projective and dense motion models wherein said data fidelity penalty term comprises the steps of:

a. estimating by a computer a blurred higher-resolution image, wherein said estimating comprises fusing said lower-resolution images; and

b. estimating by a computer a deblurred image from said blurred higher-resolution image, wherein said blurred higher-resolution image is a weighted mean of all measurements of a given pixel after zero filling and motion compensation.

3. The method according to claim 1 , wherein said data fidelity penalty term uses motion estimation errors comprising said L1 norm in a likelihood fidelity term.

4. The method according to claim 1 , wherein said spatial luminance penalty term uses bilateral-TV regularization.

5. The method according to claim 4 , wherein said bilateral TV regularization comprises a luminance image having a weighted sum of color vectors comprising red vectors, green vectors and blue vectors, a horizontal pixel-shift term, a vertical pixel-shift term, and a scalar weight between 0 and 1.

6. The method according to claim 1 , wherein said spatial chrominance penalty term uses regularization based on an L2 norm.

7. The method according to claim 1 , wherein said inter-color dependencies penalty term comprises a vector outer product norm of all pairs of neighboring pixels.

8. The method according to claim 1 , wherein using said computer to combine said data fidelity penalty term, said spatial luminance penalty term, said spatial chrominance penalty term, and said inter-color dependencies penalty term creates an overall cost function.

9. The method according to claim 8 , wherein said overall cost function comprises a steepest descent optimization, wherein said steepest optimization is applied for minimization by said computer to said overall cost function comprising the steps of:

a. applying by said computer a derivative to a first color band while having a second and a third color band held constant;

b. applying by said computer a derivative to said second color band while having said first and said third color band held constant; and

c. applying by said computer a derivative to said third color band while having said first and said second color band held constant.

10. The method according to claim 1 , wherein direct image operator effects comprising blur, high-pass filtering, masking, down-sampling, and shift are implemented by said computer in place of matrices for process speed and memory efficiency.

11. The method according to claim 1 , wherein said lower-resolution color images comprise color filtered images, compressed color images, compressed color filtered images, and an image sequence with color artifacts.

12. The method according to claim 1 , wherein said method is a computer implemented method.