IP Library Granted Patent US 10,645,291
Granted Patent B1
US 10,645,291 · App. 15/963,634 · Granted May 5, 2020

Systems and methods to mitigate adverse jitter effects in mobile imaging

Inventors: Patrick R. Mickel (San Diego, CA); Matthew C. Cristina (Oceanside, CA); Jason A. Paxton (Poway, CA)
Assignee: General Atomics Aeronautical Systems, Inc.
H04N5/2327H04N5/2328H04N5/2353H04N5/23258
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,645,291
App. No.
15/963,634
Granted
May 5, 2020
Kind
B1
Abstract

Some embodiments provide enhanced resolution imaging systems comprising: a mounting; an electro-optical image capture system; an angular jitter sensor system; an illumination source system; and an image capture control circuit is configured to: receive ling of sight displacement data; obtain, during the capture frame, an angular displacement of the image capture system and monitor when the detected angular displacement of the image capture system, based on the LOS angular displacement data, is beyond an angular displacement threshold envelope; and activate exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement of the image capture system is within the angular displacement threshold envelope, and control a level of exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement is not within the angular displacement threshold envelope.

Claims (110)

1. An enhanced resolution imaging system mounted on a moving platform, comprising:

a mounting configured to secure with a body of a platform;

an electro-optical image capture system cooperated with the mounting;

an angular jitter sensor system secured with the mounting configured to detect angular jitter noise causing line of sight (LOS) angular displacement of the image capture system;

an illumination source system cooperated with the mounting; and

an image capture control circuit communicatively coupled with the jitter sensor system, the illumination source system and the image capture system, wherein the image capture control circuit, during each image capture frame, is configured to:

receive, in real-time, LOS angular displacement data from the jitter sensor system;

obtain, during the capture frame, an angular displacement of the image capture system and monitor when the detected angular displacement of the image capture system, based on the LOS angular displacement data, is beyond an angular displacement threshold envelope; and

activate exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement of the image capture system is within the angular displacement threshold envelope, and control a level of exposure to at least decrease the exposure of the image capture system to illumination during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope;

wherein the image capture control circuit, in activating the exposure of the image capture system to the illumination, is configured to activate, during the capture frame, the illumination source system along the line of sight when the detected angular displacement of the image capture system is within the angular displacement threshold envelope to emit illumination along the line of sight; and

wherein the image capture control circuit in controlling the level of exposure is configured to adjust an integration gain of the image capture system during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope.

2. The system of claim 1 , wherein

the image capture control circuit in controlling the level of exposure is configured to deactivate the illumination source system, during the capture frame, when the detected angular displacement is not within the angular displacement threshold envelope.

3. The system of claim 2 , wherein the image capture control circuit, in activating the exposure of the image capture system to illumination, is configured to activate, during the capture frame, the image capture system to expose image sensors of the image capture system when the detected angular displacement of the image capture system is within the angular displacement threshold envelope; and

wherein the image capture control circuit in controlling the level of exposure is configured to deactivate, during the capture frame, the image capture system to prevent the imaging sensors from being exposed when the detected angular displacement is not within the angular displacement threshold envelope.

4. The system of claim 1 , wherein the image capture control circuit, in activating the exposure of the image capture system to illumination, is configured to activate, during the capture frame, the image capture system to expose image sensors of the image capture system when the detected angular displacement of the image capture system is within the angular displacement threshold envelope, and deactivate, during the capture frame, the image capture system to prevent the imaging sensors from being exposed when the detected angular displacement is not within the angular displacement threshold envelope.

5. The system of claim 4 , wherein the image capture control circuit in activating the image capture system exposes the image sensors to the illumination generated from a source external to the imaging system.

6. The system of claim 1 , wherein the image capture control circuit is configured to adjust the angular displacement threshold envelope to control illumination exposure of the image capture system.

7. The system of claim 1 , further comprising:

at least one motor secured with the mounting; and

a mechanical stabilization control circuit coupled with the motor and the jitter sensor system, wherein the stabilization control circuit is configured to control the motor to adjust an orientation of the mounting relative to the platform based on the angular displacement data from the jitter sensor system.

8. The system of claim 1 , wherein the image capture control circuit in controlling the level of exposure is configured to adjust at least one of the integration gain and illumination strength of the image capture system multiple times during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope.

9. The system of claim 1 , wherein the image capture control circuit, in activating the exposure of the image capture system to illumination, is configured to activate, multiple times during the capture frame, the image capture system to increase the integration gain to expose image sensors of the image capture system when the detected angular displacement of the image capture system is within the angular displacement threshold envelope, and deactivate by reducing the camera integration gain, multiple times during the capture frame, the image capture system to prevent the imaging sensors from being exposed when the detected angular displacement is not within the angular displacement threshold envelope.

10. An enhanced resolution imaging system mounted on a moving platform, comprising:

a mounting configured to secure with a body of a platform;

an electro-optical image capture system cooperated with the mounting;

an angular jitter sensor system secured with the mounting configured to detect angular jitter noise causing line of sight (LOS) angular displacement of the image capture system;

an illumination source system cooperated with the mounting; and

an image capture control circuit communicatively coupled with the jitter sensor system, the illumination source system and the image capture system, wherein the image capture control circuit, during each image capture frame, is configured to:

receive, in real-time, LOS angular displacement data from the jitter sensor system;

obtain, during the capture frame, an angular displacement of the image capture system and monitor when the detected angular displacement of the image capture system, based on the LOS angular displacement data, is beyond an angular displacement threshold envelope; and

activate exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement of the image capture system is within the angular displacement threshold envelope, and control a level of exposure to at least decrease the exposure of the image capture system to illumination during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope;

wherein the image capture control circuit is further configured to:

track periods of time that the illumination source system is active and emitting illumination;

estimate system limitations of the illumination source system as a function of the periods of time the illumination source system is active; and

adjust power levels of the illumination source system while active as a function of the estimated system limitations.

11. The system of claim 10 , wherein the image capture control circuit in controlling the level of exposure is configured to decrease the exposure of the image capture system to illumination multiple times during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope.

12. An enhanced resolution imaging system mounted on a moving platform, comprising:

a mounting configured to secure with a body of a platform;

an electro-optical image capture system cooperated with the mounting;

an angular jitter sensor system secured with the mounting configured to detect angular jitter noise causing line of sight (LOS) angular displacement of the image capture system;

an illumination source system cooperated with the mounting;

an image capture control circuit communicatively coupled with the jitter sensor system, the illumination source system and the image capture system, wherein the image capture control circuit, during each image capture frame, is configured to:

receive, in real-time, LOS angular displacement data from the jitter sensor system;

obtain, during the capture frame, an angular displacement of the image capture system and monitor when the detected angular displacement of the image capture system, based on the LOS angular displacement data, is beyond an angular displacement threshold envelope; and

activate exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement of the image capture system is within the angular displacement threshold envelope, and control a level of exposure to at least decrease the exposure of the image capture system to illumination during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope; and

an image processing system configured to process images captured by the image capture system and communicate image resolution data to the image capture control circuit;

wherein the image capture control circuit is configured to adjust the angular displacement threshold envelope as a function of a resulting resolution and a signal to noise ratio corresponding to a preceding frame captured by the image capture system.

13. The system of claim 12 , wherein the an image capture control circuit in activating the exposure of the image capture system to the illumination is configured to make adjustments multiple times during the capture frame to at least one of an integration gain and the exposure of the imaging system to the illumination, wherein the multiple adjustments during the capture frame correspond to times when the detected angular displacement of the image capture system is not within the angular displacement threshold envelope.

14. An enhanced resolution imaging system mounted on a moving platform, comprising:

a mounting configured to secure with a body of a platform;

an electro-optical image capture system cooperated with the mounting;

an angular jitter sensor system secured with the mounting configured to detect angular jitter noise causing line of sight (LOS) angular displacement of the image capture system;

an illumination source system cooperated with the mounting; and

an image capture control circuit communicatively coupled with the jitter sensor system, the illumination source system and the image capture system, wherein the image capture control circuit, during each image capture frame, is configured to:

receive, in real-time, LOS angular displacement data from the jitter sensor system;

obtain, during the capture frame, an angular displacement of the image capture system and monitor when the detected angular displacement of the image capture system, based on the LOS angular displacement data, is beyond an angular displacement threshold envelope; and

activate exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement of the image capture system is within the angular displacement threshold envelope, and control a level of exposure to at least decrease the exposure of the image capture system to illumination during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope;

wherein the image capture control circuit in activating the exposure of the image capture system to the illumination is configured to activate and deactivate the illumination source system multiple times per frame establishing multiple sub-integration periods of the capture frame.

15. A method of capturing enhanced resolution images from a moving platform, comprising:

by an image capture control circuit:

receiving, in real-time, line of sight (LOS) angular displacement data from a jitter sensor system secured with a mounting secured with a platform;

obtaining, during a capture frame, an angular displacement of an electro-optical image capture system secured with the mounting;

monitoring when the detected angular displacement of the image capture system, based on the LOS angular displacement data, is beyond an angular displacement threshold envelope; and

activating exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement of the image capture system is within the angular displacement threshold envelope; and

controlling a level of exposure to at least decrease the exposure of the image capture system, during the capture frame, when the detected angular displacement is not within the angular displacement threshold envelope;

wherein the activating the exposure of the image capture system to the illumination comprises activating, during the capture frame, the illumination source system along the line of sight when the detected angular displacement of the image capture system is within the angular displacement threshold envelope to emit illumination along the line of sight; and

wherein the controlling the level of exposure comprises adjusting an integration gain of the image capture system during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope.

16. The method of claim 15 ,

wherein the controlling the level of exposure comprises deactivating the illumination source system, during the capture frame, when the detected angular displacement is not within the angular displacement threshold envelope.

17. The method of claim 16 , wherein the activating the exposure of the image capture system to illumination comprises:

activating, during the capture frame, the image capture system to expose image sensors of the image capture system when the detected angular displacement of the image capture system is within the angular displacement threshold envelope; and

wherein the controlling the level of exposure comprises deactivating, during the capture frame, the image capture system to prevent the imaging sensors from being exposed when the detected angular displacement is not within the angular displacement threshold envelope.

18. The method of claim 15 , wherein the activating the exposure of the image capture system to illumination comprises:

activating, during the capture frame, the image capture system to expose image sensors of the image capture system when the detected angular displacement of the image capture system is within the angular displacement threshold envelope; and

deactivating, during the capture frame, the image capture system to prevent the imaging sensors from being exposed when the detected angular displacement is not within the angular displacement threshold envelope.

19. The method of claim 18 , wherein the activating the image capture system comprises exposing the image sensors to the illumination generated from a source external to the imaging system.

20. The method of claim 15 , further comprising:

adjusting the angular displacement threshold envelope to control illumination exposure of the image capture system.

21. The method of claim 15 , further comprising:

controlling a motor secured with the mounting and causing adjusts to an orientation of the mounting relative to the platform based on the angular displacement data from the jitter sensor system.

22. The method of claim 15 , wherein the controlling the level of exposure comprises making adjustments to at least one of the integration gain and illumination strength of the image capture system multiple times during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope.

23. A method of capturing enhanced resolution images from a moving platform, comprising:

by an image capture control circuit:

receiving, in real-time, line of sight (LOS) angular displacement data from a jitter sensor system secured with a mounting secured with a platform;

obtaining, during a capture frame, an angular displacement of an electro-optical image capture system secured with the mounting;

monitoring when the detected angular displacement of the image capture system, based on the LOS angular displacement data, is beyond an angular displacement threshold envelope; and

activating exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement of the image capture system is within the angular displacement threshold envelope;

controlling a level of exposure to at least decrease the exposure of the image capture system, during the capture frame, when the detected angular displacement is not within the angular displacement threshold envelope;

tracking periods of time that the illumination source system is active and emitting illumination;

estimating a system limitation of the illumination source system as a function of the periods of time the illumination source system is active; and

adjusting power levels of the illumination source system while active as a function of the estimated system limitation.

24. The method of claim 23 , wherein the controlling the level of exposure comprises controlling the level of the exposure to decrease the exposure of the image capture system to illumination multiple times during the capture frame when the detected angular displacement is not within the angular displacement threshold envelope.

25. A method of capturing enhanced resolution images from a moving platform, comprising:

by an image capture control circuit:

receiving, in real-time, line of sight (LOS) angular displacement data from a jitter sensor system secured with a mounting secured with a platform;

obtaining, during a capture frame, an angular displacement of an electro-optical image capture system secured with the mounting;

monitoring when the detected angular displacement of the image capture system, based on the LOS angular displacement data, is beyond an angular displacement threshold envelope; and

activating exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement of the image capture system is within the angular displacement threshold envelope;

controlling a level of exposure to at least decrease the exposure of the image capture system, during the capture frame, when the detected angular displacement is not within the angular displacement threshold envelope;

processing images captured by the image capture system and communicating image resolution data to the image capture control circuit; and

adjusting the angular displacement threshold envelope as a function of a resulting resolution and a signal to noise ratio corresponding to a preceding frame captured by the image capture system.

26. A method of capturing enhanced resolution images from a moving platform, comprising:

by an image capture control circuit:

receiving, in real-time, line of sight (LOS) angular displacement data from a jitter sensor system secured with a mounting secured with a platform;

obtaining, during a capture frame, an angular displacement of an electro-optical image capture system secured with the mounting;

monitoring when the detected angular displacement of the image capture system, based on the LOS angular displacement data, is beyond an angular displacement threshold envelope; and

activating exposure of the image capture system to illumination, during the capture frame, when the detected angular displacement of the image capture system is within the angular displacement threshold envelope; and

controlling a level of exposure to at least decrease the exposure of the image capture system, during the capture frame, when the detected angular displacement is not within the angular displacement threshold envelope;

wherein the activating the exposure of the image capture system to the illumination comprises activating and deactivating the illumination source system multiple times per frame establishing multiple sub-integration periods of the captured frame.

Assignments (5)
RELEASE OF SECURITY INTEREST Recorded Jul 14, 2025
From: BMO BANK N.A., AS ADMINISTRATIVE AGENT
To: GENERAL ATOMICS AERONAUTICAL SYSTEMS, INC.
Reel/Frame 071931/0431 →
SECURITY INTEREST Recorded Jun 16, 2025
From: GENERAL ATOMICS AERONAUTICAL SYSTEMS, INC.
To: BMO BANK N.A., SUCCESSOR BY MERGER TO BANK OF THE WEST, AS ADMINISTRATIVE AGENT
Reel/Frame 071424/0660 →
SECURITY INTEREST Recorded Jun 14, 2024
From: GENERAL ATOMICS AERONAUTICAL SYSTEMS, INC.
To: BMO BANK N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 067734/0357 →
SECURITY INTEREST Recorded Nov 5, 2019
From: GENERAL ATOMICS AERONAUTICAL SYSTEMS, INC.
To: BANK OF THE WEST
Reel/Frame 050918/0346 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 8, 2018
From: MICKEL, PATRICK R.; CRISTINA, MATTHEW C.; PAXTON, JASON A.
To: GENERAL ATOMICS AERONAUTICAL SYSTEMS, INC.
Reel/Frame 045747/0983 →
Cited By (2)
US 12,249,098 US 12,277,727