Methods using selective query recollection for enhanced training of query-based object detectors
Disclosed herein are training strategies for query-based object detectors, referred to herein as Query Recollection (QR). In one variation or QR, dense query recollection, every intermediate query is collected and independently forwarded to every downstream stage. In a second variation or QR, selective query recollection, intermediate queries are collected from the two nearest previous stages and forwarded to the next downstream stage. This eliminates the phenomena wherein intermediate stages of the decoder produce more accurate results than later stages of the decoder.
1 . A method of training a query-based object detector having a plurality of decoding stages, comprising: receiving an input query at a first stage; inputting an input query collection to each subsequent stage, the input query collection comprising an output query collection from an immediately preceding stage and a query collection used as input for the immediately preceding stage; calculating, for each subsequent stage, a loss based on the output of that stage and the input query collection for that stage; and ack-propagating the calculated loss to that stage; thereby enhancing later stages to mitigate the impact of cascading errors.
2 . The method of claim 1 further comprising:
calculating, for a last stage a loss for all intermediate queries in the output collection of the last stage of the decoder.
3 . The method of claim 1 wherein, for each stage, half of the output query collection comprises intermediate queries generated by the current stage and half of the output query collection comprises intermediate queries generated by previous stages.
4 . The method of claim 2 further comprising:
collecting an output query collection for the last stage of the decoder;
using the output query collection of the last stage of the decoder as an input query collection to the last stage; and
calculating a loss for the last stage.
5 . A method of training a query-based object detector having a plurality of decoding stages, comprising: receiving an input query at a first stage; inputting an input query collection to each subsequent stage, the input query collection for each subsequent stage comprising an output query collection from one or more preceding stages and an input query collection from the one or more preceding stages calculating, for each subsequent stage, a loss based on the output of that state and the input query collections for that stage; and back-propagating the calculated loss to that stage; thereby enhancing later stages to mitigate the impact of cascading errors.
6 . The method of claim 5 wherein the one or more previous stages comprises an immediately preceding stage.
7 . The method of claim 5 wherein the one or more previous stages comprises two immediately preceding stages.
8 . The method of claim 5 wherein the output query collection for all stages includes outputs from one or more previous stages.
9 . The method of claim 5 wherein a starting stage for including outputs from one or more previous stages in the output query collection for the current stage varies.
10 . The method of claim 5 further comprising:
calculating, for a last stage a loss for all intermediate queries in the output query collection of the last stage.