Automatic video synchronization via analysis in the spatiotemporal domain

1. A method for automatically synchronizing videos acquired by at least two cameras with overlapping views in a multi-camera network, said method comprising:

determining reference lines within an overlapping field of view of said at least two cameras in said multi-camera network, said reference lines connecting at least two pairs of corresponding points;

obtaining spatiotemporal maps of said reference lines; and

determining an optimal alignment between video segments obtained from said at least two cameras based on a registration of said spatiotemporal maps.

2. The method of claim 1 wherein said registration of said spatiotemporal maps comprises extracting features from said spatiotemporal maps and finding correspondences between the extracted features.

3. The method of claim 2 wherein said spatiotemporal maps are registered by determining parameters of a geometric transformation that best maps one spatiotemporal map to the other among said spatiotemporal maps according to the determined feature correspondences.

4. The method of claim 3 wherein said geometric transformation comprises at least one of: rotation, scaling, translation, perspective, projective, and affine.

5. The method of claim 1 further comprising registering said spatiotemporal maps via at least one of an intensity-based registration technique and a feature-based registration technique.

6. The method of claim 1 wherein said registration of said spatiotemporal maps comprises a dynamic time warping operation.

7. The method of claim 1 wherein said spatiotemporal maps are binary and wherein obtaining said spatiotemporal maps, further comprises:

performing a background estimation on said reference lines; and

based on said background estimation and incoming frames, performing a foreground detection on each of said reference lines across a period of time to obtain a binary spatiotemporal map.

8. The method of claim 1 wherein registering said spatiotemporal maps comprises:

projecting said spatiotemporal maps along a spatial dimension to obtain a temporal sequence, and

registering a resulting temporal sequence via a temporal sequence registration technique.

9. The method of claim 1 wherein corresponding points are automatically, manually or semi-automatically selected via feature extraction and matching between views among said at least two cameras.

10. The method of claim 1 wherein said at least two videos have different frame rates or wherein a temporal offset exists between said at least two videos.

11. A system for automatically synchronizing videos acquired by at least two cameras with overlapping views in a multi-camera network, said system comprising:

at least one processor; and

a computer-usable medium embodying computer program code, said computer-usable medium capable of communicating with said at least one processor, said computer program code comprising instructions executable by said at least one processor and configured for:

determining reference lines within an overlapping field of view of said at least two cameras in said multi-camera network, said reference lines connecting at least two pairs of corresponding points;

obtaining spatiotemporal maps of said reference lines; and

determining an optimal alignment between video segments obtained from said at least two cameras based on a registration of said spatiotemporal maps.

12. The system of claim 11 wherein said registration of said spatiotemporal maps comprises extracting features from said spatiotemporal maps and finding correspondences between the extracted features.

13. The system of claim 11 wherein said instructions are further configured for registering said spatiotemporal maps via at least one of an intensity-based registration technique and a feature-based registration technique.

14. The system of claim 11 wherein said registration of said spatiotemporal maps comprises a dynamic time warping operation.

15. The system of claim 11 wherein said spatiotemporal maps are registered by determining parameters of a geometric transformation that best maps one spatiotemporal map to the other among said spatiotemporal maps and wherein said geometric transformation comprises at least one of: rotation, scaling, translation, perspective, projective, and affine.

16. The system of claim 11 wherein said spatiotemporal maps are binary and wherein said instructions for obtaining said spatiotemporal maps, further comprise instructions configured for:

performing a background estimation on said reference lines; and

based on said background estimation and incoming frames, performing a foreground detection on each of said reference lines across a period of time to obtain a binary spatiotemporal map.

17. The system of claim 11 wherein registering said spatiotemporal maps comprises:

projecting said spatiotemporal maps along a spatial dimension to obtain a temporal sequence; and

registering a resulting temporal sequence via a temporal sequence registration technique.

18. The system of claim 11 wherein corresponding points are automatically, manually or semi-automatically selected via feature extraction and matching between views among said at least two cameras.

19. The system of claim 11 wherein said at least two videos have different frame rates or wherein a temporal offset exists between said at least two videos.

20. A system for automatically synchronizing videos acquired from cameras having overlapping views in a multi-camera network, said system comprising:

at least two cameras in a multi-camera network, wherein reference lines within an overlapping field of view of said at least two cameras are identified, said reference lines connecting at least two pairs of corresponding points; and

at least one processor for obtaining spatiotemporal maps of said reference lines and determining an optimal alignment between video segments obtained from said at least two cameras based on a registration of said spatiotemporal maps, wherein said at least one processor communicates with said at least two cameras.