Learning ordinal representations for deep reinforcement learning based object localization
A reinforcement learning based approach to the problem of query object localization, where an agent is trained to localize objects of interest specified by a small exemplary set. We learn a transferable reward signal formulated using the exemplary set by ordinal metric learning. It enables test-time policy adaptation to new environments where the reward signals are not readily available, and thus outperforms fine-tuning approaches that are limited to annotated images. In addition, the transferable reward allows repurposing of the trained agent for new tasks, such as annotation refinement, or selective localization from multiple common objects across a set of images. Experiments on corrupted MNIST dataset and CU-Birds dataset demonstrate the effectiveness of our approach.
1. A deep reinforcement learning (RL) method for object localization comprising:
acquiring a seed dataset including a set of seed images each with ground truth bounding box annotation;
pretrain ordinal embedding by randomly perturbing the ground truth bounding box at different levels denoted by parameter p, said ordinal embedding satisfying an ordinal constraint locally for each pair of perturbed data augmented from the same image, wherein the pretraining is performed through the effect of a backbone network, a region of interest (RoI) head, and a triplet loss; and
using an embedding function, configuring RL agents to start from a whole image and recursively sample actions from a discrete action space such that rewards are produced, the rewards of a sample action determined from embedding distances and updating a policy network based on the rewards so determined; and
outputting an annotation policy and embedding function.
2. The method of claim 1 wherein the seed image bounding box annotation is initially provided by a human action.