Pan sharpening digital imagery
View Patent ↗Method for pan sharpening a synoptic pair of multispectral images. Each multispectral image contains multiple spectral bands of M band groups, where M is a number of band groups in a range of 1 to M. A panchromatic image associated with the synoptic pair and having a p band is also provided. for each band group m in the range of 1 to M, the method includes the steps of (a) upsampling band group m; (b) pan sharpening band group m using the p band; (c) producing a pan sharpened result for band group m; and repeating steps (a)-(c) to produce pan sharpened results for each band group from 1 to M. Pan sharpened results for band groups 1 to M are fused together to produce a master fused image.
1. A method for pan sharpening a first multispectral image of an area, comprising:
providing at least a second multispectral image of the area, the first and second multispectral images comprising a synoptic pair and each multispectral image comprising multiple spectral bands of M band groups, where M is a number of band groups in a range of 1 to M;
providing a panchromatic image of the geographical area, the panchromatic image comprising a p band;
for each band group m in the range of 1 to M, with a computer:
(a) upsampling band group m;
(b) pan sharpening band group m using the p band;
(c) producing a pan sharpened result for band group m; and
continuing steps (a)-(c) to produce pan sharpened results for each band group from 1 to M; and
fusing together the pan sharpened results for band groups 1 to M to produce a master fused image.
2. The method of claim 1 , wherein the pan sharpening comprises:
smoothing the p band to produce a smoothed p band;
for each pixel, determining a vector x, the vector x having N spectral bands and comprising component xi where i is an index that runs from 1 to N;
determining an intensity based on a length of the vector x;
determining a ratio of each component xi to the intensity;
determining an adjusted component xi′ using component xi,
using the adjusted component xi′ to calculate an adjusted vector x′ to pan sharpen each pixel to produce a sharpened multispectral image.
3. The method of claim 2 , wherein determining an adjusted component xi′ comprises using component xi, the p band and the smoothed p band.
4. The method of claim 2 , wherein determining an adjusted component xi′ comprises using component xi, the p band and the intensity.
5. The method of claim 1 , wherein the pan sharpening comprises:
performing a forward transform of each of red, green and blue components for a pixel of the multispectral image to obtain hue, intensity and saturation bands, the intensity band having a level of spatial detail;
smoothing the p band to produce a smoothed p band comprising substantially the same level of spatial detail of the intensity band;
modifying the intensity band by a ratio of the p band and the smoothed p band to produce a modified intensity band;
obtaining a modified red component, a modified green component and a modified blue component using the modified intensity band; and
using the modified red, green and blue components to sharpen the multispectral image.
6. The method of claim 1 , wherein the band groups from 1 to M do not all have the same number of spectral bands.
7. The method of claim 1 , wherein the synoptic pair of multispectral images comprises multispectral images collected in separate multispectral configurations.
8. A method of pan sharpening a multispectral image of a location, comprising;
providing a panchromatic image of the location, the panchromatic image having a p band;
with a computer, performing a forward transform of each of red, green and blue components for a pixel of the multispectral image to obtain hue, intensity and saturation bands, the intensity band having a level of spatial detail;
with a computer, smoothing the p band to produce a smoothed p band comprising substantially the same level of spatial detail of the intensity band;
with a computer, modifying the intensity band using the p band and the smoothed p band to produce a modified intensity band;
obtaining a modified red component, a modified green component and a modified blue component using the modified intensity band; and
using the modified red, green and blue components to sharpen the multispectral image, producing a pan sharpened multispectral image.
9. The method of claim 8 , wherein the smoothing comprises removing high spatial frequency details from the p band.
10. The method of claim 8 , wherein the method is performed without matching a histogram of the intensity band with a histogram of the p band.
11. The method of claim 8 , wherein modifying the intensity band using the p band and the smoothed p band comprises modifying the intensity band by a ratio of the p band to the smoothed p band.
12. The method of claim 11 , wherein the modifying the intensity band by a ratio of the p band to the smoothed p band comprises multiplying the ratio of the p band to the smoothed p band by the intensity band.
13. The method of claim 8 , further comprising statistical matching by computing a mean and a standard deviation of the p band, computing a mean and a standard deviation of the intensity band and modifying the p band and the smoothed p band to match the mean and standard deviation of the intensity band.
14. The method of claim 8 , wherein the modifying the intensity band using the p band and the smoothed p band comprises modifying the intensity band by a difference between the p band and the smoothed p band.
15. The method of claim 14 , wherein the modifying by a difference between the p band and the smoothed p band comprises multiplying the difference between the p band and the smoothed band by a sharpening factor.
16. The method of claim 15 , wherein the sharpening factor is in a range of above zero to 2.
17. The method of claim 16 , wherein the sharpening factor is equal to about 1.
18. A method for pan sharpening a multispectral image of a location, the multispectral image having a plurality of pixels, comprising:
providing a panchromatic image of the location, the panchromatic image having a p band;
for each pixel, with a computer, determining a vector x, the vector x having N spectral bands and comprising component xi where i is an index that runs from 1 to N;
with a computer, determining an intensity based on a length of vector x;
with a computer, determining a ratio of component xi to the intensity;
determining an adjusted component xi′ using component xi;
using the adjusted component xi′ to calculate an adjusted vector x′ to pan sharpen each pixel to produce a sharpened multispectral image.
19. The method of claim 18 , further comprising upsampling the multispectral image.
20. The method of claim 18 , further comprising statistical matching of the p band to the intensity.
21. The method of claim 20 , wherein the statistical matching is histogram matching.
22. The method of claim 18 , wherein the method is performed without a forward transformation and a reverse transformation.
23. The method of claim 18 , wherein the determining an adjusted component xi′ using component xi comprises multiplying component xi by a ratio of the p band to the intensity.
24. The method of claim 18 , further comprising smoothing the p band to produce a smoothed p band, and wherein the determining an adjusted component xi′ using component xi comprises using component xi, the p band and the smoothed p band.
25. The method of claim 24 , wherein the determining an adjusted component xi′ comprises multiplying component xi by a ratio of the p band to the smoothed p band.
26. The method of claim 24 , wherein the determining an adjusted component xi′ comprises adding the component xi to a difference between the p band and the smoothed p band multiplied by a sharpening factor.
27. The method of claim 26 , wherein the sharpening factor is a range of above zero to 2.
28. The method of claim 27 , wherein the sharpening factor is equal to about 1.
29. A method of pan sharpening a multispectral image of a location, the multispectral image having N spectral bands, comprising;
providing a panchromatic image of the location, the panchromatic image having a p band;
with a computer, performing a forward transform of each of red, green and blue components for a pixel of the multispectral image to obtain hue, intensity and saturation bands, the intensity band having a level of spatial detail;
with a computer, generating a synthetic p band from the N spectral bands using a coefficient for each of N spectral bands based on a spectral overlap between the p band and each of the spectral bands;
with a computer, matching an intensity of the p band with an intensity of the synthetic p band, producing an intensity matched synthetic p band;
modifying the intensity band by a difference between the p band and the intensity matched synthetic p band;
obtaining a modified red component, a modified green component and a modified blue component using the modified intensity band; and
using the modified red, green and blue components to sharpen the multispectral image.
30. The method of claim 29 , wherein the modifying the intensity band comprises multiplying the difference between the p band and the smoothed band by a sharpening factor.
31. The method of claim 30 , wherein the sharpening factor is a range of above zero to 2.
32. The method of claim 31 , wherein the sharpening factor is equal to about 1.
33. A non-transitory computer-readable medium operatively associated with an image processing system for pan sharpening a first multispectral image of an area, the first multispectral image forming a synoptic pair with a second multispectral image of the area, and each multispectral image comprising multiple spectral bands of M band groups, where M is a number of band groups in a range of 1 to M, the computer readable medium including program instructions that when executed by one or more processors of the image processing system, cause the processors to carry out the steps of:
providing a panchromatic image of the area, the panchromatic image comprising a p band;
for each band group m in the range of 1 to M:
(a) upsampling band group m;
(b) pan sharpening band group m using the p band;
(c) producing a pan sharpened result for band group m; and
continuing steps (a)-(c) to produce pan sharpened results for each band group from 1 to M; and
fusing together the pan sharpened results for band groups 1 to M to produce a master fused image.
34. The computer readable medium of claim 33 , wherein the pan sharpening comprises performing steps to implement one of hue-intensity-saturation enhanced pan sharpening, hue-intensity-saturation difference modulation technique pan sharpening, or in-place pan sharpening.
35. The computer readable medium of claim 34 , wherein the pan sharpening comprises in-place pan sharpening that is performed without a forward transform and a reverse transform.
36. A non-transitory computer-readable medium operatively associated with an image processing system for pan sharpening a multispectral image of a location, the multispectral image having a plurality of pixels, the computer readable medium including program instructions that when executed by one or more processors of the image processing system, cause the processors to carry out the steps of:
providing a panchromatic image of the location, the panchromatic image having a p band;
upsampling the multispectral image;
for each pixel, determining a vector x, the vector x having N spectral bands and comprising component xi where i is an index that runs from 1 to N;
determining an intensity based on a length of vector x;
determining a ratio of component xi to the intensity;
determining an adjusted component xi′ using component xi;
using the adjusted component xi′ to calculate an adjusted vector x′ to pan sharpen each pixel to produce a sharpened multispectral image.
37. The computer-readable medium of claim 36 , wherein the steps are performed without a forward transformation and a reverse transformation.
38. The computer-readable medium of claim 36 , wherein the steps comprise smoothing the p band to produce a smoothed p band, and wherein the determining an adjusted component xi′ using component xi comprises adding the component xi to a difference between the p band and the smoothed p band multiplied by a sharpening factor.
39. The computer-readable medium of claim 38 , wherein the sharpening factor is a range of above zero to 2.
40. A non-transitory computer-readable medium operatively associated with an image processing system for pan sharpening a multispectral image of a location, the multispectral image having N bands, the computer readable medium including program instructions that when executed by one or more processors of the image processing system, cause the processors to carry out the steps of:
providing a panchromatic image of the location, the panchromatic image having a p band;
performing a forward transform of each of red, green and blue components for a pixel of the multispectral image to obtain hue, intensity and saturation bands, the intensity band having a level of spatial detail;
generating a synthetic p band from the N spectral bands using a coefficient for each of N spectral bands based on a spectral overlap between the p band and each of the spectral bands;
matching an intensity of the p band with an intensity of the synthetic p band, producing an intensity matched synthetic p band;
modifying the intensity band by a difference between the p band and the intensity matched synthetic p band;
obtaining a modified red component, a modified green component and a modified blue component using the modified intensity band; and
using the modified red, green and blue components to sharpen the multispectral image.
41. The computer-readable medium of claim 40 , wherein the modifying the intensity band comprises multiplying the difference between the p band and the smoothed band by a sharpening factor.
42. The computer-readable medium of claim 41 , wherein the sharpening factor is a range of above zero to 2.