IP Library Granted Patent US 7,236,646
Granted Patent B1
US 7,236,646 · App. 10/423,720 · Granted Jun 26, 2007

Tonal balancing of multiple images

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,236,646
App. No.
10/423,720
Granted
Jun 26, 2007
Kind
B1
Abstract

Methods are disclosed for tonally balancing multiple images. In one embodiment, the method comprises using a subset of corresponding points in each of a plurality of image overlap regions to solve a set of minimization equations for gains and biases for each spectral band of each image, the corresponding points being points from different images having locations that correspond to each other, the subset of corresponding points consisting of corresponding points whose intensities differ less than a threshold, applying the gains and biases to the images, and iterating said using and applying actions for a predetermined number of iterations.

Claims (46)

1. A method for tonally balancing a set of overlapping images, comprising:

using a subset of corresponding points in each of a plurality of image overlap regions to solve a set of minimization equations for gains and biases for each spectral band of each image, the corresponding points being points from different images having locations that correspond to each other, the subset of corresponding points consisting of corresponding points whose intensities differ less than a threshold;

applying the gains and biases to the images;

iterating said using and applying actions for a predetermined number of iterations, said iterating resulting in an iterated gain and an iterated bias for each spectral band of each image;

transforming intensities of points for each spectral band in each image using the iterated gain and the iterated bias for the corresponding spectral band to produce transformed intensities of points; and

producing a tonally balanced image mosaic by using the transformed intensities of points in said images.

2. A method of claim 1 , wherein the set of minimization equations penalize gains that stray from a desired gain.

3. A method of claim 2 , wherein the desired gain is 1.

4. A method of claim 2 , wherein the set of minimization equations encourage gains for a common spectral band to average to the desired gain.

5. A method of claim 1 , wherein the set of minimization equations penalize biases that stray from a desired bias.

6. A method of claim 5 , wherein the desired bias is 0.

7. A method of claim 1 , wherein the set of minimization equations encourage the gains for the spectral bands of a particular image to move together for the particular image.

8. A method of claim 7 , wherein the set of minimization equations encourage the biases for the spectral bands of the particular image to move together for the particular image.

9. A method of claim 8 , wherein the encouragement for the gains of the particular image to move together is stronger than the encouragement for the biases of the particular image to move together.

10. A method of claim 1 , wherein the set of minimization equations weight corresponding points based on a size of the overlap in which they reside, to thereby give differing sizes of image overlaps equal importance in said minimization equations.

11. A method of claim 1 , wherein the subset of corresponding points differs for each iteration of the using and applying actions.

12. A method of claim 11 , wherein the threshold increases or decreases for each iteration of said using and applying actions, depending on a size of the subset during a previous iteration of the using and applying actions.

13. A method of claim 1 , wherein the subset of corresponding points does not include points having a value in a two-dimensional histogram of corresponding pairs of points below a second threshold.

14. A method of claim 1 , further comprising, adjusting the biases of one or more images uniformly to minimize clipping of intensities of the images.

15. A method of claim 1 , wherein the biases of all images are adjusted uniformly to match a worst case underclipping and worst case overclipping of intensities of the images.

16. The method of claim 1 , further comprising before using the subset of corresponding points, applying an atmospheric correction to the points.

17. The method of claim 16 , wherein applying an atmospheric correction comprises:

for a first spectral band of the first image, creating a histogram of a frequency of point intensities in the first image;

equating the atmospheric correction to a lowest intensity that both i) has a frequency greater than a threshold frequency, and ii) is within a defined intensity range of a predetermined number of additional intensities that exceed said threshold frequency.

18. The method of claim 1 , further comprising before using the subset of corresponding points, applying a backscattering correction to the points.

19. The method of claim 1 , further comprising calculating the threshold by:

for the first iteration, setting the threshold equal to a predetermined value; and

for subsequent iterations, dividing the last threshold by a multiplier if the number of points in the subset is more than a minimum value, and multiplying the last threshold by the multiplier if the number of points in the subset is less than the minimum value.

20. The method of claim 1 , wherein the set of minimization equations implement a linear least squares minimization.

21. The method of claim 20 , wherein the linear least squares minimization includes an overlap term, the overlap term summing a difference squared of transformed intensities of the subset of corresponding points, the transformed intensities including parameters for the gains and parameters for the biases.

22. The method of claim 21 , further comprising applying an atmospheric correction to the transformed intensities.

23. The method of claim 21 , further comprising applying a backscattering correction to the transformed intensities.

24. A method comprising:

tonally balancing a set of at least three images, the set having at least a first overlap region between a first pair of images, and a second overlap region between a second pair of images, the images being tonally balanced by:

identifying intensities of a set of corresponding points in each of a plurality of image overlap regions, the corresponding points being points from different images having locations that correspond to each other;

applying an algorithm using the intensities to obtain a gain and a bias for each spectral band of each image, the gains and the biases being chosen to minimize differences between corresponding points in the overlap regions after the gains and the biases have been applied; and

transforming intensities of points for each spectral band in said images using the gain and bias for the corresponding spectral band; and

producing a tonally balanced image mosaic from the tonally balanced set of at least three images.

25. The method of claim 24 , further comprising before transforming the intensities, eliminating at least some corresponding points from the set.

26. The method of claim 25 , wherein eliminating at least some corresponding points comprises eliminating corresponding points when the corresponding points have a difference in intensity greater than a threshold.

27. The method of claim 25 , wherein eliminating at least some corresponding points comprises eliminating corresponding points having a value in a two-dimensional histogram of corresponding pairs of points below a threshold.

28. The method of claim 25 , further comprising iterating the eliminating, and applying for a predetermined number of iterations.

29. The method of claim 28 , wherein the gains for the first iteration are one and the biases for the first iteration are zero.

30. The method of claim 28 , wherein the gains and biases for subsequent iterations comprise the gains and biases obtained by the last iteration.

31. The method of claim 28 , wherein the algorithm comprises at least one of a first weighting term to encourage the gains to average to a desired gain, a second weighting term to encourage the biases to average to a desired bias, a third weighting term to encourage the gains for the spectral bands of a particular image to move together for the particular image, and a fourth weighting term to encourage the biases for the spectral bands of the particular image to move together for the particular image.

32. The method of claim 31 , further comprising, after iterating for the predetermined number of iterations, if one or more of the obtained gains fall outside a desired range, adjusting one or more of the weights and repeating the iterating the eliminating, transforming, and applying for the predetermined number of iterations.

Assignments (14)
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 5, 2023
From: MAXAR INTELLIGENCE INC. (F/K/A DIGITALGLOBE, INC.); AURORA INSIGHT INC.; MAXAR MISSION SOLUTIONS INC. ((F/K/A RADIANT MISSION SOLUTIONS INC. (F/K/A THE RADIANT GROUP, INC.)); MAXAR SPACE LLC (F/K/A SPACE SYSTEMS/LORAL, LLC); SPATIAL ENERGY, LLC; MAXAR SPACE ROBOTICS LLC ((F/K/A SSL ROBOTICS LLC) (F/K/A MDA US SYSTEMS LLC)); MAXAR TECHNOLOGIES HOLDINGS INC.
To: SIXTH STREET LENDING PARTNERS, AS ADMINISTRATIVE AGENT
Reel/Frame 063660/0138 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS AND TRADEMARKS - RELEASE OF REEL/FRAME 051258/0465 Recorded May 4, 2023
From: ROYAL BANK OF CANADA, AS AGENT
To: MAXAR INTELLIGENCE INC.; MAXAR SPACE LLC
Reel/Frame 063542/0300 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS AND TRADEMARKS - RELEASE OF REEL/FRAME 044167/0396 Recorded May 4, 2023
From: ROYAL BANK OF CANADA, AS AGENT
To: MAXAR INTELLIGENCE INC.; MAXAR SPACE LLC
Reel/Frame 063543/0001 →
TERMINATION AND RELEASE OF PATENT SECURITY AGREEMENT - RELEASE OF REEL/FRAME 053866/0412 Recorded May 4, 2023
From: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
To: MAXAR INTELLIGENCE INC.; MAXAR SPACE LLC
Reel/Frame 063544/0011 →
RELEASE OF SECURITY INTEREST Recorded Jun 21, 2022
From: WILMINGTON TRUST, NATIONAL ASSOCIATION
To: DIGITALGLOBE, INC.; SPACE SYSTEMS/LORAL, LLC; RADIANT GEOSPATIAL SOLUTIONS LLC
Reel/Frame 060390/0282 →
PATENT SECURITY AGREEMENT Recorded Sep 23, 2020
From: DIGITALGLOBE, INC.
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 053866/0412 →
SECURITY AGREEMENT (NOTES) Recorded Dec 12, 2019
From: DIGITALGLOBE, INC.; RADIANT GEOSPATIAL SOLUTIONS LLC; SPACE SYSTEMS/LORAL, LLC (F/K/A SPACE SYSTEMS/LORAL INC.)
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, - AS NOTES COLLATERAL AGENT
Reel/Frame 051262/0824 →
AMENDED AND RESTATED U.S. PATENT AND TRADEMARK SECURITY AGREEMENT Recorded Dec 11, 2019
From: DIGITALGLOBE, INC.
To: ROYAL BANK OF CANADA, AS COLLATERAL AGENT
Reel/Frame 051258/0465 →
SECURITY INTEREST Recorded Oct 5, 2017
From: DIGITALGLOBE, INC.; MACDONALD, DETTWILER AND ASSOCIATES LTD.; MACDONALD, DETTWILER AND ASSOCIATES CORPORATION; MACDONALD, DETTWILER AND ASSOCIATES INC.; MDA GEOSPATIAL SERVICES INC.; SPACE SYSTEMS/LORAL, LLC; MDA INFORMATION SYSTEMS LLC
To: ROYAL BANK OF CANADA, AS THE COLLATERAL AGENT
Reel/Frame 044167/0396 →
RELEASE OF SECURITY INTEREST IN PATENTS FILED AT R/F 041069/0910 Recorded Oct 5, 2017
From: BARCLAYS BANK PLC
To: DIGITALGLOBE, INC.
Reel/Frame 044363/0524 →
SECURITY INTEREST Recorded Jan 23, 2017
From: DIGITALGLOBE, INC.
To: BARCLAYS BANK PLC, AS COLLATERAL AGENT
Reel/Frame 041069/0910 →
MERGER Recorded Jan 5, 2017
From: GEOEYE SOLUTIONS HOLDCO INC.
To: DIGITALGLOBE, INC.
Reel/Frame 040866/0571 →
MERGER Recorded Jan 5, 2017
From: GEOEYE SOLUTIONS INC.
To: GEOEYE SOLUTIONS HOLDCO INC.
Reel/Frame 040866/0490 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (029734/0427) Recorded Dec 22, 2016
From: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
To: DIGITALGLOBE INC.; GEOEYE ANALYTICS INC.; GEOEYE, LLC; GEOEYE SOLUTIONS INC.
Reel/Frame 041176/0066 →