Non-linear transformations for enhancement of images
View Patent ↗Imaging apparatus ( 26 ) is provided for use with an image sensor ( 24 ). The apparatus includes a non-linear mapping circuit ( 42 ), which is configured to receive a raw stream of input pixel values generated by the image sensor and to perform a non-linear mapping of the input pixel values. to generate a mapped stream of mapped pixel values, and a linear convolution filter ( 44 ), which is arranged to filter the mapped stream of mapped pixel values to generate a filtered stream of filtered pixel values. Other embodiments are also described.
1. Imaging apparatus, comprising:
a first non-linear mapping circuit, which is configured to receive a stream of input pixel values and to implement a first non-linear mapping function to map the input pixel values to a mapped stream of mapped pixel values, wherein the input pixel values have an input blur caused by a point spread function (PSF) of a set of objective optics, wherein the input pixel values include a first subset of low-amplitude pixel values having a first range of amplitude values, and wherein the first non-linear mapping function maps the first subset of low-amplitude pixel values to a first corresponding subset of mapped pixel values having a first mapped range of amplitude values that is greater than the first range of amplitude values, such that the first non-linear mapping function causes the first subset of low-amplitude pixel values to be spread out over a larger range of mapped pixel values; and
a linear convolution filter, which is configured to filter the mapped stream of mapped pixel values to generate a filtered stream of filtered pixel values, wherein the linear convolution filter comprises a deconvolution filter, which has a filter kernel determined according to the PSF, and which is configured to filter the mapped pixel values to cause the filtered pixel values to have less blur than the input pixel values.
2. The apparatus according to claim 1 , wherein the first non-linear mapping circuit is configured to implement the first non-linear mapping function to map each of the input pixel values individually, irrespective of any neighboring input pixel values.
3. The apparatus according to claim 1 , wherein the raw stream of input pixel values includes at least 100,000 input pixel values, and wherein the first non-linear mapping circuit is configured to implement the first non-linear mapping function such that at least 10% of the mapped pixel values are different from the corresponding input pixel values.
4. The apparatus according to claim 1 , further comprising:
a second non-linear mapping circuit, which is configured to receive the filtered stream of filtered pixel values and to implement a second non-linear mapping function to map the filtered pixel values to an output stream of output pixel values, wherein the second non-linear mapping function is an inverse of the first non-linear mapping function for at least a range of allowable filtered pixel values.
5. The apparatus according to claim 1 , wherein the first non-linear mapping function includes a power law transformation, and wherein the first non-linear mapping circuit is configured to perform the power law transformation.
6. The apparatus according to claim 1 , wherein the first non-linear mapping function comprises a piecewise linear function, and wherein the first non-linear mapping circuit is configured to implement the piecewise linear function.
7. The apparatus according to claim 6 , wherein the first non-linear mapping circuit is configured to use curve approximation to smooth knees in the piecewise linear function.
8. The apparatus according to claim 6 , wherein the piecewise linear function includes a plurality of segments, and wherein the first non-linear mapping circuit comprises a sparse lookup table having respective entries that specify the segments.
9. The apparatus according to claim 1 , wherein the stream of input pixel values is provided by a mosaic image sensor, and wherein the stream of input pixel values belongs to a plurality of interleaved input sub-images, each input sub-image responsive to light of a different, respective color that is incident on the mosaic image sensor, and wherein the first non-linear mapping circuit is configured to generate a plurality of interleaved output sub-images respectively, and wherein the apparatus further comprises an image signal processor (ISP), which is coupled to receive and combine the plurality of output sub-images in order to generate a color video output image.
10. The apparatus according to claim 1 , wherein the first non-linear mapping circuit is configured to dynamically alter the first non-linear mapping function responsively to one or more characteristics of the input pixel values.
11. The apparatus according to claim 1 , wherein the raw stream of input pixel values further includes a second subset of high-amplitude pixel values having a second range of amplitude values, and wherein the first non-linear mapping function maps the second subset of high-amplitude pixel values to a second corresponding subset of the mapped pixel values having a second mapped range of amplitude values that is smaller than the second range of amplitude values such that the first non-linear mapping function causes the second subset of high-amplitude pixel values to be compressed into a smaller range of mapped pixel values.
12. A method for imaging, comprising:
receiving a stream of input pixel values, wherein the input pixel values have an input blur caused by a point spread function (PSF) of a set of objective optics, and wherein the input pixel values include a first subset of low-amplitude pixel values having a first range of amplitude values;
performing a first non-linear mapping function to map the input pixel values to a mapped stream of mapped pixel values, wherein the first non-linear mapping function maps the first subset of low-amplitude pixel values, having the first range of amplitude values, to a first corresponding subset of mapped pixel values having a first mapped range of amplitude values that is greater than the first range of amplitude values, such that the first non-linear mapping function causes the first subset of low-amplitude pixel values to be spread out over a larger range of mapped pixel values; and
applying a linear convolution filter to the mapped stream of mapped pixel values to generate a filtered stream of filtered pixel values, wherein the linear convolution filter comprises a deconvolution filter, which has a filter kernel determined according to the PSF, and which filters the mapped pixel values to cause the filtered pixel values to have less blur than the input pixel values.
13. The method according to claim 12 , wherein performing the first non-linear mapping function comprises performing the first non-linear mapping function to map each of the input pixel values individually, irrespective of any neighboring input pixel values.
14. The method according to claim 12 , wherein the stream of input pixel values includes at least 100,000 of the input pixel values, and wherein performing the first non-linear mapping function comprises performing the first non-linear mapping function such that at least 10% of the mapped pixel values are different from the corresponding input pixel values.
15. The method according to claim 12 , further comprising:
performing a second non-linear mapping function to map the filtered pixel values to an output stream of output pixel values, wherein the second non-linear mapping function is an inverse of the first non-linear mapping function for at least a range of allowable filtered pixel values.
16. The method according to claim 12 , wherein performing the first non-linear mapping function comprises performing a power law transformation.
17. The method according to claim 12 , wherein performing the first non-linear mapping function comprises performing a piecewise linear mapping function.
18. The method according to claim 17 , wherein performing the piecewise linear mapping function comprises using curve approximation to smooth knees in the piecewise linear mapping function.
19. The method according to claim 17 , wherein performing the piecewise linear mapping function comprises using a sparse lookup table having respective entries that specify a plurality of piecewise linear segments.
20. The method according to claim 12 , wherein the stream of input pixel values is provided by a mosaic image sensor, wherein the stream of input pixel values belongs to a plurality of interleaved input sub-images, each input sub-image responsive to light of a different, respective color that is incident on the mosaic image sensor, wherein performing the first non-linear mapping function comprises generating a plurality of interleaved output sub-images respectively, and wherein the method further comprises receiving and combining the plurality of output sub-images in order to generate a color video output image.
21. The method according to claim 12 , wherein performing the first non-linear mapping function comprises dynamically altering the first non-linear mapping function responsively to one or more characteristics of the input pixel values.
22. The method according to claim 12 , wherein the raw stream of input pixel values further includes a second subset of relatively high-amplitude pixel values having a second range of amplitude values, and wherein performing the first non-linear mapping function comprises mapping the second subset of high-amplitude pixel values to a second corresponding subset of the mapped pixel values having a second mapped range of amplitude values that is smaller than the second range of amplitude values, such that the first non-linear mapping function causes the second subset of high-amplitude pixel values to be compressed into a smaller range of mapped pixel values.