Apparatus and method for shift invariant differential (SID) image data interpolation in non-fully populated shift invariant matrix
View Patent ↗An image processing system interpolates image data of an image array by ascertaining shift invariant points and non-shift invariant points within the array. The average illumination and the second order derivative are determined for the shift invariant locations. The second order derivative and the intensity at the non-shift invariant locations for each of the non-shift invariant points are estimated. The color data for each color element is determined from the image data and second order derivative. The second order derivative is multiplied by a scaling factor for selectively smoothing and sharpening the second order derivative. The color data values of adjacent color element to enhance a resolution of the image data.
1. A system comprising:
an array of color sensors including shift invariant points and non-shift invariant points at interstitial spaces between the color sensors; and
a processing device configured to determine illumination values associated with the shift invariant points based on image data from the color sensors that surround the shift invariant points, to estimate illumination values associated with the non-shift invariant points based, at least in part, on the illumination values associated with the shift invariant points that surround the non-shift invariant points, and to determine color data for color sensors corresponding to each interstitial space based, at least in part, on the illumination values associated with the shift invariant points and the non-shift invariant points.
2. The system of claim 1 , wherein the processing device is configured to determine the locations of shift invariant points and non-shift invariant points in the array of color sensors from geometric data corresponding to a configuration of the array of color sensors.
3. The system of claim 1 , wherein the processing device is configured to average the image data from the color sensors that surround corresponding shift invariant points to determine the illumination values associated with the shift invariant points.
4. The system of claim 1 , wherein the processing device is configured to determine second order derivatives of the illumination values corresponding to the shift invariant points, and to determine the color data corresponding to the shift invariant points based, at least in part, on the second order derivatives of the illumination values corresponding to the shift invariant points.
5. The system of claim 4 , wherein the processing device is configured to average the illumination values that surround each shift invariant point to determine the second order derivatives of the illumination values corresponding to the shift invariant points.
6. The system of claim 4 , wherein the processing device is configured to selectively smooth or sharpen the second order derivatives of the illumination values corresponding to the shift invariant points based on a scaling factor.
7. The system of claim 4 , wherein the processing device is configured to estimate second order derivatives of the illumination values corresponding to the non-shift invariant points based, at least in part, on the second order derivatives corresponding to the shift invariant points.
8. The system of claim 7 , wherein the processing device is configured to average the second order derivatives of the illumination values corresponding to the shift invariant points that surround each shift non-invariant point to estimate the second order derivatives of the illumination values corresponding to the non-shift invariant points.
9. The system of claim 7 , wherein the processing device is configured to estimate the illumination values associated with the non-shift invariant points based, at least in part, on the illumination values associated with the shift invariant points that surround the non-shift invariant points and the estimated second order derivatives of the illumination values corresponding to the non-shift invariant points.
10. The system of claim 1 , wherein the processing device is configured to determine the color data corresponding to non-shift invariant points based, at least in part, on the second order derivatives of the illumination values corresponding to the non-shift invariant points.
11. A method comprising:
determining, by an imaging device, illumination values associated with shift invariant points at interstitial spaces of an array of color sensors based on image data from color sensors that surround the shift invariant points;
estimating, by the imaging device, illumination values associated with non-shift invariant points at interstitial spaces of the array of color sensors based, at least in part, on the illumination values associated with the shift invariant points that surround the non-shift invariant points; and
determining, by the imaging device, color data for color sensors corresponding to each interstitial space based, at least in part, on the illumination values associated with the shift invariant points and the non-shift invariant points.
12. The method of claim 11 , further comprising:
determining, by the imaging device, the locations of shift invariant points and non-shift invariant points in the array of color sensors from geometric data corresponding to a configuration of the array of color sensors.
13. The method of claim 11 , wherein determining the illumination values associated with the shift invariant points further comprises averaging, by the imaging device, the image data from the color sensors that surround corresponding shift invariant points.
14. The method of claim 11 , further comprising:
determining, by the imaging device, second order derivatives of the illumination values corresponding to the shift invariant points; and
determining, by the imaging device, the color data corresponding to the shift invariant points based, at least in part, on the second order derivatives of the illumination values corresponding to the shift invariant points.
15. The method of claim 14 , wherein determining the second order derivatives of the illumination values corresponding to the shift invariant points further comprises averaging, by the imaging device, the illumination values that surround each shift invariant point.
16. The method of claim 14 , further comprising selectively smoothing or sharpening, by the imaging device, the second order derivatives of the illumination values corresponding to the shift invariant points based on a scaling factor.
17. The method of claim 14 , further comprising estimating, by the imaging device, second order derivatives of the illumination values corresponding to the non-shift invariant points based, at least in part, on the second order derivatives corresponding to the shift invariant points.
18. The method of claim 17 , wherein estimating the second order derivatives of the illumination values corresponding to the non-shift invariant points further comprises averaging, by the imaging device, the second order derivatives of the illumination values corresponding to the shift invariant points that surround each shift non-invariant point.
19. The method of claim 17 , wherein estimating the illumination values associated with the non-shift invariant points is based, at least in part, on the illumination values associated with the shift invariant points that surround the non-shift invariant points and the estimated second order derivatives of the illumination values corresponding to the non-shift invariant points.
20. The method of claim 11 , wherein determining the color data corresponding to non-shift invariant points is based, at least in part, on the second order derivatives of the illumination values corresponding to the non-shift invariant points.
21. An apparatus comprising a computer-readable memory device storing instructions configured to cause a processing device to perform operations comprising:
determining illumination values associated with shift invariant points at interstitial spaces of an array of color sensors based on image data from color sensors that surround the shift invariant points;
estimating illumination values associated with non-shift invariant points at interstitial spaces of the array of color sensors based, at least in part, on the illumination values associated with the shift invariant points that surround the non-shift invariant points; and
determining color data for color sensors corresponding to each interstitial space based, at least in part, on the illumination values associated with the shift invariant points and the non-shift invariant points.
22. The apparatus of claim 21 , wherein the instructions are configured to cause the processing device to perform operations further comprising determining the locations of shift invariant points and non-shift invariant points in the array of color sensors from geometric data corresponding to a configuration of the array of color sensors.
23. The apparatus of claim 21 , wherein the instructions are configured to cause the processing device to perform operations further comprising averaging the image data from the color sensors that surround corresponding shift invariant points.
24. The apparatus of claim 21 , wherein the instructions are configured to cause the processing device to perform operations further comprising:
determining second order derivatives of the illumination values corresponding to the shift invariant points; and
determining the color data corresponding to the shift invariant points based, at least in part, on the second order derivatives of the illumination values corresponding to the shift invariant points.
25. The apparatus of claim 24 , wherein the instructions are configured to cause the processing device to perform operations further comprising averaging the illumination values that surround each shift invariant point.
26. The apparatus of claim 24 , wherein the instructions are configured to cause the processing device to perform operations further comprising selectively smoothing or sharpening the second order derivatives of the illumination values corresponding to the shift invariant points based on a scaling factor.
27. The apparatus of claim 24 , wherein the instructions are configured to cause the processing device to perform operations further comprising estimating second order derivatives of the illumination values corresponding to the non-shift invariant points based, at least in part, on the second order derivatives corresponding to the shift invariant points.
28. The apparatus of claim 27 , wherein the instructions are configured to cause the processing device to perform operations further comprising averaging the second order derivatives of the illumination values corresponding to the shift invariant points that surround each shift non-invariant point.
29. The apparatus of claim 27 , wherein the instructions are configured to cause the processing device to perform operations further comprising estimating the illumination values associated with the non-shift invariant points based, at least in part, on the illumination values associated with the shift invariant points that surround the non-shift invariant points and the estimated second order derivatives of the illumination values corresponding to the non-shift invariant points.
30. The apparatus of claim 21 , wherein the instructions are configured to cause the processing device to perform operations further comprising determining the color data corresponding to non-shift invariant points based, at least in part, on the second order derivatives of the illumination values corresponding to the non-shift invariant points.
31. A system comprising:
means for determining illumination values associated with shift invariant points at interstitial spaces of an array of color sensors based on image data from color sensors that surround the shift invariant points;
means for estimating illumination values associated with non-shift invariant points at interstitial spaces of the array of color sensors based, at least in part, on the illumination values associated with the shift invariant points that surround the non-shift invariant points; and
means for determining color data for color sensors corresponding to each interstitial space based, at least in part, on the illumination values associated with the shift invariant points and the non-shift invariant points.
32. The system of claim 31 , further comprising means for determining the locations of shift invariant points and non-shift invariant points in the array of color sensors from geometric data corresponding to a configuration of the array of color sensors.
33. The system of claim 31 , wherein the means for determining the illumination values associated with the shift invariant points is further configured to average the image data from the color sensors that surround corresponding shift invariant points.
34. The system of claim 31 , further comprising means for determining second order derivatives of the illumination values corresponding to the shift invariant points, wherein the means for determining color data is configured to determine the color data corresponding to the shift invariant points based, at least in part, on the second order derivatives of the illumination values corresponding to the shift invariant points.
35. The system of claim 34 , wherein the means for determining the second order derivatives of the illumination values corresponding to the shift invariant points is configured to average the illumination values that surround each shift invariant point.
36. The system of claim 34 , further comprising means for selectively smoothing or sharpening the second order derivatives of the illumination values corresponding to the shift invariant points based on a scaling factor.
37. The system of claim 34 , further comprising means for estimating second order derivatives of the illumination values corresponding to the non-shift invariant points based, at least in part, on the second order derivatives corresponding to the shift invariant points.
38. The system of claim 37 , wherein the means for estimating the second order derivatives of the illumination values corresponding to the non-shift invariant points is further configured to average the second order derivatives of the illumination values corresponding to the shift invariant points that surround each shift non-invariant point.
39. The system of claim 37 , wherein the means for estimating the illumination values associated with the non-shift invariant points is configured to estimate the illumination values associated with the non-shift invariant points based, at least in part, on the illumination values associated with the shift invariant points that surround the non-shift invariant points and the estimated second order derivatives of the illumination values corresponding to the non-shift invariant points.
40. The system of claim 31 , wherein the means for determining the color data is further configured to determine the color intensity data corresponding to non-shift invariant points based, at least in part, on the second order derivatives of the illumination values corresponding to the non-shift invariant points.