Edge hardware online model learning and range estimation for autonomous aimpoint determination

1 . A platform configured to intercept a target, the platform comprising:

a guidance system comprising a seeker configured to generate an aimpoint corresponding to a specific location on the target; and

a control system configured to receive the aimpoint from the guidance system and control a position of at least a portion of the platform, relative to the target, and to bring the at least a portion of the platform onto an intercept course with the target,

wherein the seeker comprises:

an image sensor configured to capture image data associated with a forward-looking perspective of the platform, the image data comprising image frames potentially containing the target; and

a processor in communication with the image sensor and with a non-transitory storage medium comprising instructions which, when executed by the processor, cause the processor to:

receive the image data associated with the forward-looking perspective of the platform being directed to the target, the image data comprising the image frames potentially containing the target;

apply a first method of image processing to the image data, the first method of image processing being one that does not require the use of a target model, to finalize a pixel area in at least one image frame that matches characteristics associated with the target, to generate at least one track point corresponding to a distinctive portion of the target, and to provide the aimpoint associated with the target to the control system;

while iteratively applying the first method of image processing to the image data:

store the at least one image frame that matches characteristics associated with the target thereby creating a reference image frame containing the target;

determine the at least one track point on the target, the at least one track point comprising an invariant feature associated with the target;

describe the at least one track point identified in the reference image frame using track point descriptors, wherein the reference image frame, the at least one track point, and the track point descriptors comprise the target model;

search for and update the at least on track point in a subsequent image frame;

search for new track points in the subsequent image frame; and

describe the new track points in the subsequent image frame using the track point descriptors and create an updated target model using the new track points;

apply a second method of image processing to the image data, the second method of image processing comprising using the updated target model to track the target and updating the aimpoint; and

provide the updated aimpoint to the control system.

2 . The platform according to claim 1 , wherein the processor is further configured to determine a range to the target and/or size of the target using only the image data, wherein said determination of the range to target and/or the size of target is based on a rate of reduction of target size using a linear regression problem and wherein the processor is further configured to model a target angle in pixels (α pix ) using range rate (ΔR), instantaneous field of view ( I FOV), time to go (ttg), and cross-range projected dimension(s), as:

∝

p

⁢

i

⁢

x

=

tan

-

1

(

s

ttg

*

Δ

⁢

R

)

IFOV

to use the model to solve for time to go, and then to combine the model with a range rate to determine the range to the target.

3 . The platform according to claim 1 , wherein the first method of image processing is selected from the group consisting of image segmentation, blob detection, corner detection, and Hough transforms.

4 . The platform according to claim 1 , wherein the image sensor is selected from the group consisting of infrared imagers, electro-optical imagers, radio telescopes, tomographic scanners, and optical imagers.

5 . The platform according to claim 1 , wherein a threshold quality value is applied to track points and used to determine if the seeker will lock onto those track points satisfying the threshold quality value and wherein the threshold value is recalculated per frame and is dependent on image quality.

6 . The platform according to claim 1 , wherein the aimpoint comprises an offset from the at least one track point.

7 . The platform according to claim 6 , wherein the offset is an equation comprising coefficients or values that can be set prior to execution of the instructions by the processor and/or mutate thereafter.

8 . The platform according to claim 1 , wherein the processor is configured to switch from the first method of image processing to the second method of image processing upon a detection of clipping or upon a pixel spatial resolution falling below a predetermined value.

9 . The platform according to claim 1 , wherein the processor is configured to compare target size and image frame size, to calculate when clipping is expected, and to transition from the first image processing method to the second image processing method before clipping is expected to occur, and wherein an image frame obtained prior to when clipping is expected to occur is used as the basis for the target model for all subsequent image processing.

10 . The platform according to claim 1 , wherein the processor is configured to receive information regarding current or anticipated directional changes and to prioritize searching for track points on a side of an image frame that is expected to remain visible longest, based on a direction of a turn.

11 . The platform according to claim 1 , wherein the first method of image processing comprises:

using an image segmentation method, searching a first image frame for a single component blob within predetermined size limits corresponding to the target; and

generating an estimated location for the target and searching for the single component blob within predetermined size limits corresponding to the target in a later image frame using the previously estimated location for the target, and

updating the estimated location for the target.

12 . The platform of claim 11 , wherein, if multiple, single component blobs within predetermined size limits corresponding to the target are found in the current image frame, the single component blob closest to a cue is selected and used for generating the estimated location for the target.

13 . A non-transitory computer readable medium comprising instructions that, when executed by a processor, cause the processor to:

receive image data associated with a forward-looking perspective of a platform being directed to a target, the image data comprising image frames potentially containing the target;

apply a first method of image processing to the image data, the first method of image processing being one that does not require the use of a target model, to finalize a pixel area in at least one image frame that matches characteristics associated with the target, to generate at least one track point corresponding to a distinctive portion of the target, and to provide an aimpoint associated with the target to a control system configured to control a position of at least a portion of the platform, relative to the target, and to bring the at least a portion of the platform onto an intercept course with the target;

while iteratively applying the first method of image processing to the image data:

store the at least one image frame that matches characteristics associated with the target thereby creating a reference image frame containing the target;

determine the at least one track point on the target, the at least one track point comprising an invariant feature associated with the target;

describe the at least one track point identified in the reference image frame using track point descriptors, the reference image frame, the at least one track point, and the track point descriptors comprising the target model;

search for and update the at least one track point in a subsequent image frame;

search for new track points in the subsequent image frame; and

describe the new track points in the subsequent image frame using the track point descriptors and create an updated target model using the new track points;

apply a second method of image processing to the image data, the second method of image processing comprising using the updated target model to track the target and update the aimpoint; and

provide the updated aimpoint to the control system.

14 . The computer readable medium according to claim 13 , wherein the processor is further configured to estimate a range to the target and/or size of the target using only the image data, wherein said determination of the range to target and/or the size of target is based on a rate of reduction of target size using a linear regression problem and wherein the processor is further configured to model a target angle in pixels (α pix ) using range rate (ΔR), instantaneous field of view ( I FOV), time to go (ttg), and cross-range projected dimension(s), as:

∝

p

⁢

i

⁢

x

=

tan

-

1

(

s

ttg

*

Δ

⁢

R

)

IFOV

to use the model to solve for time to go, and then to combine the model with a range rate to determine the range to the target.

15 . The computer readable medium according to claim 13 , wherein the aimpoint is offset from the at least one track point.

16 . The computer readable medium according to claim 15 , wherein the offset comprises an equation comprising coefficients or values that can be set prior to execution of the instructions stored on the computer readable medium by the processor and/or mutate thereafter.

17 . The computer readable medium according to claim 13 , wherein the processor is configured to switch from the first method of image processing to the second method of image processing upon a detection of clipping or upon a pixel spatial resolution falling below a threshold value.

18 . The computer readable medium according to claim 13 , wherein the processor is configured to compare target size and image frame size, to calculate when clipping is expected, and to transition from the first image processing method to the second image processing method before clipping is expected to occur, and wherein an image frame obtained prior to when clipping is expected to occur is used as the basis for the target model for all subsequent image processing.

19 . The computer readable medium according to claim 13 , wherein the processor is configured to receive information regarding current or anticipated directional changes of the platform and to prioritize searching for track points on a side of an image frame that is expected to remain visible longest, based on the current or anticipated directional changes.

20 . A seeker configured to seek a target, the seeker being configured to generate an aimpoint corresponding to a specific location on the target and comprising:

an image sensor configured to capture image data associated with a forward-looking perspective of a platform being directed to the target, the image data comprising image frames potentially containing the target; and

a processor in communication with the image sensor and with a non-transitory storage medium comprising instructions configured to cause the processor to:

receive the image data;

apply a first method of image processing to the image data, the first method of image processing being one that does not require the use of a target model, to finalize a pixel area in at least one image frame that matches characteristics associated with the target, to generate at least one track point corresponding to a distinctive portion of the target, and to generate the aimpoint associated with the target and provide the aimpoint to a control system configured to control a position of at least a portion of the platform, relative to the target, and to bring the at least a portion of the platform onto an intercept course with the target;

while iteratively applying the first method of image processing to the image data:

store the at least one image frame thereby creating a reference image frame containing the target;

determine the at least one track point on the target, the at least one track point comprising an invariant feature associated with the target;

describe the at least one track point identified in the reference image frame using track point descriptors, the reference image frame, the at least one track point, and the track point descriptors comprising the target model;

search for and update the at least one track point in a subsequent image frame;

search for new track points in the subsequent image frame; and

describe the new track points in the subsequent image frame using the track point descriptors and create an updated target model using the new track points;

apply a second method of image processing to the image data, the second method of image processing comprising using the target model to track the target and update the aimpoint; and

estimate a range to the target and/or size of the target using only the image data, wherein said determination of the range to target and/or the size of target is based on a rate of reduction of target size using a linear regression problem,

wherein the processor is configured to model a target angle in pixels (α pix ) using range rate (ΔR), instantaneous field of view (I FOV ), time to go (ttg), and cross-range projected dimension(s), as:

∝

p

⁢

i

⁢

x

=

tan

-

1

(

s

ttg

*

Δ

⁢

R

)

IFOV

to use the model to solve for time to go, and then to combine the model with a range rate to determine the range to the target.