IP Library Granted Patent US 11,563,899
Granted Patent B2
US 11,563,899 · App. 16/993,663 · Granted Jan 24, 2023

Parallelization technique for gain map generation using overlapping sub-images

Inventor: Matthew J. Penn (Tucson, AZ)
Assignee: Raytheon Company
H04N5/243G06T5/50H04N5/2258G06T2207/20021G06T2207/20216
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,563,899
App. No.
16/993,663
Granted
Jan 24, 2023
Kind
B2
Abstract

A method includes obtaining multiple spatially-displaced input images of a scene based on image data captured using multiple imaging sensors. The method also includes dividing each of the input images into multiple overlapping sub-images. The method further includes generating multiple overlapping sub-image gain maps based on the sub-images. In addition, the method includes combining the sub-image gain maps to produce a final gain map identifying relative gains of the imaging sensors. An adjacent and overlapping pair of sub-image gain maps are combined by renormalizing gain values in at least one of the pair of sub-image gain maps so that average gain values in overlapping regions of the pair of sub-image gain maps are equal or substantially equal.

Claims (52)

1. A method comprising:

obtaining multiple spatially-displaced input images of a scene based on image data captured using multiple imaging sensors, at least two of the multiple imaging sensors having different gains;

dividing each of the input images into multiple overlapping sub-images;

generating multiple overlapping sub-image gain maps based on the sub-images, wherein each sub-image gain map is generated using the sub-images in same divided positions within each of the input images and different sub-image gain maps are generated using the sub-images in different divided positions within each of the input images; and

combining the sub-image gain maps to produce a final gain map identifying relative gains of the imaging sensors with respect to one another, wherein an adjacent and overlapping pair of sub-image gain maps are combined by renormalizing gain values in at least one of the pair of sub-image gain maps so that average gain values in overlapping regions of the pair of sub-image gain maps are equal or substantially equal.

2. The method of claim 1 , wherein generating the multiple overlapping sub-image gain maps comprises generating the multiple overlapping sub-image gain maps in parallel using multiple processors or multiple processing cores.

3. The method of claim 1 , wherein combining the sub-image gain maps comprises:

combining the pair of sub-image gain maps to produce a first intermediate gain map; and

combining the first intermediate gain map with a third sub-image gain map to produce a second intermediate gain map by renormalizing gain values in at least one of the first intermediate gain map and the third sub-image gain map.

4. The method of claim 3 , further comprising:

renormalizing gain values in a last intermediate gain map to produce the final gain map.

5. The method of claim 1 , wherein:

each sub-image gain map is normalized to have an average gain value equal to a specified value; and

at least one of the average gain values in the overlapping regions of the pair of sub-image gain maps is not equal to the specified value prior to the renormalization.

6. The method of claim 1 , wherein the overlapping sub-images comprise overlapping rectangular sub-images.

7. The method of claim 1 , wherein boundaries of the sub-images overlap by between eight to sixteen pixels.

8. An apparatus comprising:

at least one memory configured to store multiple spatially-displaced input images of a scene based on image data captured using multiple imaging sensors, at least two of the multiple imaging sensors having different gains; and

at least one processor configured to:

divide each of the input images into multiple overlapping sub-images;

generate multiple overlapping sub-image gain maps based on the sub-images; and

combine the sub-image gain maps to produce a final gain map identifying relative gains of the imaging sensors with respect to one another;

wherein the at least one processor is configured to combine an adjacent and overlapping pair of sub-image gain maps by renormalizing gain values in at least one of the pair of sub-image gain maps so that average gain values in overlapping regions of the pair of sub-image gain maps are equal or substantially equal; and

wherein the at least one processor is configured to generate each sub-image gain map using the sub-images in same divided positions within each of the input images and to generate different sub-image gain maps using the sub-images in different divided positions within each of the input images.

9. The apparatus of claim 8 , wherein the at least one processor comprises multiple processors, multiple processing cores, or both configured to generate the multiple overlapping sub-image gain maps in parallel.

10. The apparatus of claim 8 , wherein, to combine the sub-image gain maps, the at least one processor is configured to:

combine the pair of sub-image gain maps to produce a first intermediate gain map; and

combine the first intermediate gain map with a third sub-image gain map to produce a second intermediate gain map by renormalizing gain values in at least one of the first intermediate gain map and the third sub-image gain map.

11. The apparatus of claim 10 , wherein the at least one processor is further configured to renormalize gain values in a last intermediate gain map to produce the final gain map.

12. The apparatus of claim 8 , wherein:

the at least one processor is configured to normalize each sub-image gain map to have an average gain value equal to a specified value; and

at least one of the average gain values in the overlapping regions of the pair of sub-image gain maps is not equal to the specified value prior to the renormalization.

13. The apparatus of claim 8 , wherein the overlapping sub-images comprise overlapping rectangular sub-images.

14. The apparatus of claim 8 , wherein boundaries of the sub-images overlap by between eight to sixteen pixels.

15. A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:

obtain multiple spatially-displaced input images of a scene based on image data captured using multiple imaging sensors, at least two of the multiple imaging sensors having different gains;

divide each of the input images into multiple overlapping sub-images;

generate multiple overlapping sub-image gain maps based on the sub-images; and

combine the sub-image gain maps to produce a final gain map identifying relative gains of the imaging sensors with respect to one another;

wherein the instructions that when executed cause the at least one processor to combine the sub-image gain maps comprise:

instructions that when executed cause the at least one processor to renormalize gain values in at least one of a pair of sub-image gain maps so that average gain values in overlapping regions of the pair of sub-image gain maps are equal or substantially equal; and

wherein the instructions that when executed cause the at least one processor to generate the sub-image gain maps comprise:

instructions that when executed cause the at least one processor to generate each sub-image gain map using the sub-images in same divided positions within each of the input images and to generate different sub-image gain maps using the sub-images in different divided positions within each of the input images.

16. The non-transitory computer readable medium of claim 15 , wherein the instructions that when executed cause the at least one processor to combine the sub-image gain maps further comprise:

instructions that when executed cause the at least one processor to:

combine the pair of sub-image gain maps to produce a first intermediate gain map; and

combine the first intermediate gain map with a third sub-image gain map to produce a second intermediate gain map by renormalizing gain values in at least one of the first intermediate gain map and the third sub-image gain map.

17. The non-transitory computer readable medium of claim 16 , further containing instructions that when executed cause the at least one processor to renormalize gain values in a last intermediate gain map to produce the final gain map.

18. The non-transitory computer readable medium of claim 15 , further containing instructions that when executed cause the at least one processor to normalize each sub-image gain map to have an average gain value equal to a specified value;

wherein at least one of the average gain values in the overlapping regions of the pair of sub-image gain maps is not equal to the specified value prior to the renormalization.

19. The non-transitory computer readable medium of claim 15 , wherein the overlapping sub-images comprise overlapping rectangular sub-images.

20. The non-transitory computer readable medium of claim 15 , wherein boundaries of the sub-images overlap by between eight to sixteen pixels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 14, 2020
From: PENN, MATTHEW J.
To: RAYTHEON COMPANY
Reel/Frame 053498/0079 →
Continuity (1)
Related Publication 20220053144A1 · Feb 17, 2022