Tracking camera and operation thereof
A tracking camera is disclosed, including a mirror assembly and sensors communicatively coupled to a controller. The controller receives data from the sensors and adjusts the mirror assembly based on the data. The sensors include a time-of-flight sensor for determining distance.
1 . A tracking camera comprising:
a mirror assembly;
a hybrid sensor that includes a camera sensor and a Time of Flight (TOF) sensor, the hybrid sensor including pixels configured to determine the time of flight and the camera sensor includes an event camera; and
a control circuitry
configured to:
receive event data from the camera sensor and time of flight data from the TOF sensor,
estimate a bearing array corresponding to a tracked object based on the event data and the time of flight data, and
adjust the mirror assembly based on the bearing array.
2 . The tracking camera of claim 1 , further comprising:
a lens assembly for collecting light for the TOF sensor and the camera sensor.
3 . The tracking camera of claim 2 , wherein:
the lens assembly includes a first subassembly for collecting light for the camera sensor and a second subassembly for collecting light for the TOF sensor.
4 . The tracking camera of claim 1 , wherein
the camera sensor further includes a frame camera, and the camera sensor data further includes image data.
5 . The tracking camera of claim 1 , wherein
the mirror assembly comprises an actuatable mirror configured for adjustment based on an adjustment signal from the control circuitry.
6 . The tracking camera of claim 5 , wherein
the actuatable mirror of the mirror assembly is a galvanometer mirror, a microelectromechanical mirror, or a piezoelectric mirror.
7 . The tracking camera of claim 5 , wherein
the control circuitry is configured to:
determine the adjustment signal based on the data, and
transmit the adjustment signal to the mirror assembly for adjusting the mirror assembly.
8 . The tracking camera of claim 1 , further comprising
an orientation feedback system communicatively coupled to the control circuitry for communicating a mirror assembly status.
9 . The tracking camera of claim 1 , wherein
the control circuitry is configured to:
transmit the adjustment signal such that an optical axis of the tracking camera which extends out of the tracking camera is moved to a target direction based on the bearing array.
10 . The tracking camera of claim 1 , further comprising:
a beam splitter configured to split light from a field of view to each of the camera sensor and the TOF sensor.
11 . The tracking camera of claim 1 , further comprising:
a laser for irradiating an object of interest and for providing a TOF signal for the TOF sensor.
12 . The tracking camera of claim 11 , further comprising:
a beam splitter intersecting an optical axis of the tracking camera and configured to direct light from the laser out of the tracking camera.
13 . A method of operating a tracking camera, comprising:
receiving data from a hybrid sensor that includes a Time of Flight (TOF) sensor and a camera sensor, the hybrid sensor including pixels configured to determine the time of flight and the camera sensor data includes event data;
estimating a bearing array corresponding to a tracked object based on the event data and time of flight data from the TOF sensor,
determining an adjustment signal based on the bearing array;
transmitting the adjustment signal to a mirror assembly; and
adjusting the mirror assembly according to the adjustment signal.
14 . The method of operating a tracking camera of claim 13 , further comprising:
forming an image of the tracked object on the camera sensor.
15 . The method of operating a tracking camera of claim 13 , further comprising:
accumulating the data for a time interval;
determining a common time within the time interval; wherein
the bearing array includes an element at the common time based on the accumulated data.
16 . A non-transitory computer-readable storage medium storing a program code that, when executed by a processor, causes the processor to execute a method of operating a tracking camera, comprising:
receiving data from a hybrid sensor that includes a Time of Flight (TOF) sensor and a camera sensor, wherein the hybrid sensor includes pixels configured to determine the time of flight and camera sensor data includes event data;
estimating a bearing array corresponding to a tracked object based on the event data and time of flight data from the TOF sensor, determining an adjustment signal based on the bearing array;
transmitting the adjustment signal to a mirror assembly; and
adjusting the mirror assembly according to the adjustment signal.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein the processor is caused to:
receive an image of a tracked object on the camera sensor,
estimating a bearing array based on the data, the bearing array corresponding to the tracked object; and
determine the adjustment signal is based on the bearing array.
18 . The non-transitory computer-readable storage medium of claim 17 , wherein the processor is caused to:
accumulate the data for a time interval; and
determine a common time within the time interval, wherein
the bearing array includes an element at the common time based on the accumulated data.