Multimodal imager systems and methods with steerable fields of view
Systems and methods related to steerable fields of view are provided. In one embodiment, a system includes an image sensor configured to capture a field of view. The system further includes an optical assembly including an optical element configured to direct the field of view to the image sensor. The system further includes a logic circuit. The logic circuit may send a control signal to the optical assembly to cause rotation of the optical element about an axis associated with the image sensor to adjust the field of view. The logic circuit may send a control signal to the image sensor to cause one or more images of the object to be captured. The logic circuit may calibrate the image sensor based on the one or more images of the object. Related devices and methods are also provided.
1 . A system comprising:
an image sensor configured to capture a field of view;
an optical assembly comprising an optical element configured to direct the field of view to the image sensor;
a housing configured to enclose at least the image sensor, the optical assembly, and a calibration object; and
a logic circuit configured to:
send a control signal to the optical assembly to cause rotation of the optical element about an axis associated with the image sensor to adjust the field of view;
send a control signal to the image sensor to cause one or more images of the adjusted field of view to be captured, wherein the adjusted field of view at least partially encompasses the calibration object; and
calibrate the image sensor based on the one or more images.
2 . The system of claim 1 , further comprising:
a radar configured to send and receive radar signals; and
wherein the logic circuit is configured to:
receive the radar signals from the radar; and
based on the radar signals, send a control signal to the optical assembly to cause rotation of the optical element to adjust the field of view.
3 . The system of claim 1 , wherein:
the system further comprises:
a radar configured to send and receive radar signals; and
the housing is configured to enclose at least the image sensor, the optical assembly, the calibration object, and the radar.
4 . The system of claim 1 , wherein:
the control signal sent to the optical assembly causes the optical assembly to rotate the optical element about the axis to adjust the field of view in an oscillating fashion; and
the logic circuit is configured to:
capture, via the image sensor, images of the oscillating field of view; and
combine the images of the oscillating field of view to provide a panoramic image of a scene.
5 . The system of claim 4 , wherein:
the logic circuit is configured to operate the system in a first mode and a second mode;
the first mode comprises rotation of the optical element about the axis in the oscillating fashion and the capturing the field of view via the image sensor as the optical element is rotated about the axis; and
the second mode comprises placing the optical element in a stationary position in response to received radar signals and capturing the field of view via the image sensor when the optical element is in the stationary position.
6 . The system of claim 1 , wherein:
the optical element comprises a mirror; and
the optical assembly further comprises an actuator configured to receive the control signal from the logic circuit to rotate the mirror about the axis to adjust the field of view to the image sensor.
7 . The system of claim 1 , wherein the calibration object comprises an internal surface of the housing.
8 . The system of claim 1 , wherein:
the calibration object spans less than the field of view such that subsets of detectors of the image sensor sequentially sense the calibration object as the optical element is rotated about the axis to adjust the field of view.
9 . The system of claim 1 , wherein:
the calibration object comprises a reverse-side of the optical element; and
the logic circuit is configured to:
send the control signal to the optical assembly to cause the rotation of the optical element such that the reverse-side of the optical element faces the image sensor and such that the adjusted field of view at least partially encompasses the reverse-side of the optical element.
10 . The system of claim 1 , further comprising:
a radar configured to send and receive radar signals within a scene;
wherein:
the one or more images capture a target in the scene; and
the logic circuit is configured to:
classify the target in the scene based on the one or more images that capture the target and radar return signals reflected by the target and received by the radar.
11 . The system of claim 1 , wherein:
the control signal causes the optical assembly to rotate the optical element to adjust the field of view of the image sensor by sub-pixel increments; and
the logic circuit is configured to generate a super resolution image based on a combination of images captured via the image sensor as the field of view is adjusted by the sub-pixel increments.
12 . A system comprising:
an image sensor configured to capture a field of view;
an optical assembly comprising an optical element configured to direct the field of view to the image sensor;
a logic circuit configured to:
send a control signal to the optical assembly to cause rotation of the optical element about an axis associated with the image sensor to adjust the field of view;
send a control signal to the image sensor to cause one or more images of the adjusted field of view to be captured;
determine a frame-to-frame motion between a first image and a second image of the one or more images based on a motion of the adjustment of the field of view;
compare identical scene coordinates of the first image and the second image based on the frame-to-frame motion to determine pixel intensity differences at the identical scene coordinates; and
calibrate the image sensor based on the one or more images and the pixel intensity differences.
13 . The system of claim 12 , further comprising a housing configured to enclose at least the image sensor, the optical assembly, and a calibration object.
14 . The system of claim 12 , further comprising:
a radar configured to send and receive radar signals; and
a housing configured to enclose at least the image sensor, the optical assembly, and the radar.
15 . A method comprising:
receiving radar signals from a radar, the radar signals reflected from a target in a scene;
based on the radar signals, steering an optical element such that a field of view directed by the optical element to an image sensor is adjusted to encompass the target, wherein the steering comprises switching from rotating the optical element in an oscillating fashion to placing the optical element in a stationary position to provide the adjusted field of view;
capturing, via the image sensor, one or more images of the target; and
classifying the target in the scene based on images of the field of view captured by the image sensor and the radar signals reflected by the target and received by the radar.
16 . The method of claim 15 , further comprising:
rotating the optical element about an axis associated with the image sensor to adjust the field of view to encompass at least a portion of an object;
capturing, via the image sensor, one or more images of the object; and
calibrating the image sensor based on the one or more images of the object.
17 . The method of claim 15 , further comprising:
rotating the optical element about an axis associated with the image sensor to adjust the field of view in an oscillating fashion;
capturing, via the image sensor, images of the oscillating field of view; and
combining the images of the oscillating field of view to provide a panoramic image.
18 . The method of claim 15 , wherein:
the steering the optical element to adjust the field of view of the image sensor is performed in sub-pixel increments;
the one or more images of the target are captured, via the image sensor, as the field of view is adjusted by the sub-pixel increments; and
the method further comprises generating a super resolution image based on a combination of the one or more images captured as the field of view is adjusted by the sub-pixel increments.
19 . The method of claim 15 , wherein the steering the optical element is in response to a classification of the target, the method further comprising:
training a radar-based classifier corresponding to the radar based on ground truth data obtained from an imager classifier corresponding to the image sensor, wherein the ground truth data comprises a classification of the target based on the one or more images.
20 . The method of claim 15 , further comprising:
training an imager classifier corresponding to the image sensor based on ground truth data obtained from a radar-based classifier corresponding to the radar, wherein the ground truth data comprises a classification of the target based on the radar signals reflected from the target in the scene.