Method and apparatus to extend the effective dynamic range of an image sensing device and use residual images

1. An image capture system for generating and storing an extended dynamic range digital image, comprising:

a) a image sensing device having fast photosites with a predetermined response to light exposure interspersed with slow photosites with a slower response to the same light exposure for producing a sparsely sampled high resolution digital image having fast pixel values produced by the fast photosites and slow pixel values produced by the slow photosites;

b) a digital image processor that employs the slow pixel values to expand the dynamic range of the fast pixel values in the sparsely sampled high resolution digital image to form a full resolution digital image having an extended dynamic range;

c) an encoder for reducing the dynamic range of the full resolution digital image to fit within the dynamic range of a storage space having dynamic range less than the dynamic range of the full resolution digital image to form a limited dynamic range digital image represented in the storage space and for producing a residual image representing a difference between the full resolution digital image and the limited dynamic range digital image that can be used with the limited dynamic range digital image to reconstruct the full resolution digital image; and

d) a digital image store for storing the limited dynamic range digital image in association with the residual image.

2. The image capture system claimed in claim 1 , wherein the image sensing device is a color image sensor having an array of photosites and a color filter array arranged over the array of photosites; wherein the digital image processor includes a color filter array interpolator for interpolating color values at all of the photosites; and wherein the storage space is a storage color space.

3. The image capture system claimed in claim 1 , wherein the digital image processor also employs the fast pixel values to expand the dynamic range of the slow pixel values in the sparsely sampled high resolution digital image.

4. The image capture system claimed in claim 1 , wherein the image sensing device is located in a digital camera and the digital image processor is located in a host computer separate from the digital camera.

5. The image capture system claimed in claim 1 , wherein the image sensing device and the digital image processor are included in a digital camera.

6. The image capture system claimed in claim 1 , wherein the slow photosites have a response that is slower by at least one stop compared to the fast photosites.

7. The system claimed in claim 2 , wherein the color filter array is a Bayer array.

8. The image capture system claimed in claim 1 , further comprising an image reconstructor that employs the residual image and the limited dynamic range digital image to form a reconstructed full resolution digital image.

9. The image capture system claimed in claim 8 , wherein the image reconstructor applies an image modification to the reconstructed fill resolution digital image.

10. The image capture system claimed in claim 1 , wherein the limited dynamic range digital image is stored by the image store in a digital image file which includes the residual image as additional data in the digital image file.

11. The image capture system as claimed in claim 1 , wherein the limited dynamic range digital image and the residual image are stored by the image store in separate associated digital image files.

12. A method for generating and storing an extended dynamic range digital image, comprising:

a) employing an image sensing device having fast photosites with a predetermined response to light exposure interspersed with slow photosites with a slower response to the same light exposure to produce a sparsely sampled high resolution digital image having fast pixel values produced by the fast photosites and slow pixel values produced by the slow photosites;

b) expanding the dynamic range of the fast pixel values in the sparsely sampled high resolution digital image to form a full resolution digital image having an extended dynamic range;

c) reducing the dynamic range of the full resolution digital image to fit within the dynamic range of a storage space having a dynamic range less than the dynamic range of the full resolution digital image to form a limited dynamic range digital image represented in the storage space and for producing a residual image representing a difference between the full resolution digital image and the limited dynamic range digital image that can be used with the limited dynamic range digital image to reconstruct the full resolution digital image; and

d) storing the limited dynamic range digital image in association with the residual image.

13. The method as claimed in claim 12 , further comprising the step of expanding the dynamic rang of the slow pixel values by utilizing neighboring fast pixel values.

14. The method as claimed in claim 13 , wherein the step of expanding the respective response of the fast and slow pixel values, comprises the steps of:

a) comparing fast pixel values against a high exposure response threshold and replacing the fast pixel values exceeding the high exposure response threshold with pixel values that are a function of neighboring blow pixel values; and

b) comparing slow pixel values against a low exposure response threshold and replacing the slow pixel values less than the low exposure response threshold with pixel values that are a function of neighboring fast pixel values.

15. The method claimed in claim 12 , wherein a residual image representing a luminance difference and a residual image representing a chrominance difference are produced and stored.

16. The method claimed in claim 12 , wherein the residual image is stored as tiles representing subset of pixels in separate tags in an image file.

17. An image capture system for generating and storing an extended dynamic range digital image, comprising:

a) a image sensing device having fast photosites with a predetermined response to light exposure interspersed with slow photosites with a slower response to the same light exposure for producing a sparsely sampled high resolution digital image having fast pixel values produced by the fast photosites and slow pixel values produced by the slow photosites;

b) a digital image processor that employs the slow pixel values to expand the dynamic range of the fast pixel values in the sparsely sampled high resolution digital image to form a full resolution digital image having extended dynamic range;

c) an encoder for reducing the dynamic range of the full resolution digital image to fit within the dynamic range of a storage space having a dynamic range less than the dynamic range of the full resolution digital image to form limited dynamic range digital image represented in the storage space and for producing a residual image representing a difference between the sparsely sampled high resolution digital image and the limited dynamic range digital image that can be used with the limited dynamic range digital image to reconstruct the sparsely sampled high resolution digital image; and

d) a digital image store for storing the limited dynamic range digital image in association with the residual image.