Learned Dynamic Camera System Control for Human-Pose Estimation

To get optimal camera images for human pose estimation, including, specifically, hand tracking, a network is trained to simultaneously do hand pose estimation and camera control. By combining these tasks into a single network, the accuracy of the hand tracking during training is used as feedback to guide how the network controls the camera parameters. This approach is enhanced by independently controlling the exposure parameters of each participating camera or sensor. This expands the dynamic range beyond what is possible with a single camera, enabling improved functionality across a broader range of environments or with lower bit depths and reduced system power. This method is applicable to systems with any number of tracking sensors, as it involves capturing multi-exposure images of the scene volume both temporally and spatially.