Occlusion-aware multi-object tracking
A system for tracking a target object across a plurality of image frames. The system comprises a logic machine and a storage machine. The storage machine holds instructions executable by the logic machine to calculate a trajectory for the target object over one or more previous frames occurring before a target frame. Responsive to assessing no detection of the target object in the target frame, the instructions are executable to predict an estimated region for the target object based on the trajectory, predict an occlusion center based on a set of candidate occluding locations for a set of other objects within a threshold distance of the estimated region, each location of the set of candidate occluding locations overlapping with the estimated region, and automatically estimate a bounding box for the target object in the target frame based on the occlusion center.
1. A system for tracking a target object across a plurality of image frames, comprising:
a logic machine; and
a storage machine holding instructions executable by the logic machine to:
calculate a trajectory for the target object over one or more previous frames occurring before a target frame, wherein the target object is detected by tracking, in a similarity matrix, comparison values indicating similarity between object feature data for a first set of objects detected in a first previous frame, the first set of objects including the target object, and object feature data for a second set of objects in a second previous frame, wherein the similarity matrix includes:
a row for each object in a union of both of the first set of objects and the second set of objects; and
a column for each object in the union,
wherein each matrix element of the similarity matrix represents one comparison value between a pair of objects drawn from the union;
responsive to assessing no detection of the target object in the target frame:
upon determining that the target object is not detected in the target frame due to being occluded by a set of one or more other objects, predict an estimated region for the target object based on the trajectory;
predict an occlusion center based on a set of candidate occluding locations for the set of other objects within a threshold distance of the estimated region, each location of the set of candidate occluding locations overlapping with the estimated region; and
automatically estimate a bounding box for the target object in the target frame based on the occlusion center, wherein the bounding box is estimated via a trained machine learning system trained via supervised learning with image data and ground-truth bounding boxes.
2. The system of claim 1 , wherein the instructions are further executable to calculate a heatmap for the occlusion center by operating one or more convolutional neural network units.
3. The system of claim 1 , wherein estimating the bounding box includes operating a state machine configured to record state information including tracking status and tracked motion of a plurality of objects, and to estimate the bounding box based on such recorded state information.
4. The system of claim 1 , wherein estimating the bounding box includes operating a Kalman filter.
5. The system of claim 1 , further comprising a training storage device holding instructions executable to train a machine learning system to track objects based on one or more unsupervised labels.
6. The system of claim 5 , wherein the similarity matrix further includes
a placeholder column for the placeholder unsupervised labels.
7. The system of claim 5 , wherein the object feature data and the comparison values are generated by one or more differentiable functions, and wherein training the machine learning system to track objects includes configuring the one or more differentiable functions based on the unsupervised labels and one or more automatic supervision signals.
8. A method of tracking a target object across a plurality of image frames, the method comprising:
calculating a trajectory for the target object over one or more previous frames occurring before a target frame, wherein the target object is detected by tracking, in a similarity matrix, comparison values indicating similarity between object feature data for a first set of objects detected in a first previous frame, the first set of objects including the target object, and object feature data for a second set of objects in a second previous frame, wherein the similarity matrix includes:
a row for each object in a union of both of the first set of objects and the second set of objects; and
a column for each object in the union,
wherein each matrix element of the similarity matrix represents one comparison value between a pair of objects drawn from the union;
assessing no detection of the target object in the target frame due to the target object being occluded by a set of one or more other objects;
predicting an estimated region for the target object based on the trajectory;
predicting an occlusion center based on a set of candidate occluding locations for the set of other objects within a threshold distance of the estimated region, each location of the set of candidate occluding locations overlapping with the estimated region; and
automatically estimating a bounding box for the target object in the target frame based on the occlusion center, wherein the bounding box is estimated via a trained machine learning system trained via supervised learning with image data and ground-truth bounding boxes.
9. The method of claim 8 , further comprising calculating a heatmap for the occlusion center by operating one or more convolutional neural network units.
10. The method of claim 8 , wherein estimating the bounding box includes operating a state machine configured to record state information including tracking status and tracked motion of a plurality of objects, and to estimate the bounding box based on such recorded state information.
11. The method of claim 8 , wherein estimating the bounding box includes operating a Kalman filter.