Surveillance System and Method for Tracking and Identifying Objects in Environments
A method and system tracks objects using a surveillance database storing events acquired by a set of sensors and sequences of images acquired by a set of cameras. Sequences of temporally and spatially adjacent events sensed by the set of sensors are linked to form a set of tracklets and stored in the database. Each tracklet has endpoints being either a track-start, track-join, tracklet-merge or tracklet-end node. A subset of sensors is selected, and a subset of tracklets associated with the subset of sensors is identified. A single starting tracklet is selected. All sequences of tracklets temporally and spatially adjacent to the starting tracklet are aggregated to construct a tracklet graph. The track-join nodes and the track-split nodes are disambiguated and eliminated from the track graph to determine a track of the object in the environment.
1 . A computer implemented method for tracking objects using a surveillance database, the surveillance database storing events acquired by a set of sensors and sequences of images acquired by a set of cameras, each event and image having an associated location and time, the method comprising the steps of:
linking sequences of temporally and spatially adjacent events sensed by the set of sensors to form a set of tracklets, each tracklet beginning with a track-start node, a track-join node or a tracklet-split node and ending with a track-end node, the tracklet-join node or the tracklet-split node, the tracklet-join nodes occurring where multiple preceding tracklets merge to a single successor tracklet and the track-split nodes occurring where a single preceding tracklet diverges to multiple successor tracklets;
selecting a subset of sensors;
identifying a subset of tracklets associated with the subset of sensors
selecting a single tracklet from the subset of tracklet as a starting tracklet;
aggregating all tracklets temporally and spatially adjacent to the starting tracklet to construct a tracklet graph; and
disambiguating and eliminating the track-join nodes and the track-split nodes from the tracklet graph to determine a track of an object in the environment.
2 . The method of claim 1 , in which the disambiguating further comprising:
displaying available images temporally and spatially related to the events of the tracklet graph to identify the object.
3 . The method of claim 1 , in which the sensors are infra-red motion sensors, and the cameras are movable.
4 . The method of claim 1 , in which the sensors using wireless transmitters for transmitting the events.
5 . The method of claim 1 , further comprising:
retrieving the sequences of images only when events are detected by sensors in a view of a particular camera.
6 . The method of claim 5 , further comprising:
directing the particular camera at a general vicinity of the particular sensor when a particular event is sensed.
7 . The method of claim 1 , in which the aggregating is performed according to temporal and spatial constraints.
8 . The method of claim 8 , in which the temporal and spatial constraints are selected by a user.
9 . The method of claim 8 , in which the temporal and spatial constraints are learned over time.
10 . The method of claim 1 , further comprising:
drawing the track on a floor plan of the environment.
11 . The method of claim 1 , further comprising:
associating particular sequences of images with the tracklets.
12 . The method of claim 11 , further comprising:
collecting the particular sequences of images associated with the track as video evidence related to the track and object.
13 . The method of claim 1 , further comprising:
identifying sensors with cameras at any given time.
14 . The method of claim 1 , further comprising:
identifying particular events visible in the sequences of images at any given time.
15 . The method of claim 14 , further comprising:
reducing the video evidence to only images corresponding to visible sensor activations.
16 . The method of claim 1 , in which the linking step is performed periodically and the set of tracklets are pre-stored in the surveillance database.
17 . A system for tracking objects using a surveillance database, the surveillance database storing events acquired by a set of sensors and sequences of images acquired by a set of cameras, each event and image having an associated location and time, the system comprising:
means for linking sequences of temporally and spatially adjacent events sensed by the set of sensors to form a set of tracklets, each tracklet beginning with a track-start node, a track-join node or a tracklet-split node and ending with a track-end node, the tracklet-join node or the tracklet-split node, the tracklet-join nodes occurring where multiple preceding tracklets merge to a single successor tracklet and the track-split nodes occurring where a single preceding tracklet diverges to multiple successor tracklets;
means for selecting a staring tracklet;
a user interface selecting a subset of sensors;
means for aggregating all tracklets temporally and spatially adjacent to the starting tracklet to construct a tracklet graph; and
means for disambiguating and eliminating the track-join nodes and the track-split nodes from the tracklet graph to determine a track of an object in the environment.
18 . The system of claim 17 , in which the disambiguating further comprises:
means for displaying available images temporally and spatially related to the events of the tracklet graph to identify the object.
19 . The system of claim 18 , in which the sensors are infra-red motion sensors, and the cameras are movable.