Automatic camera calibration and geo-registration using objects that provide positional information

1. A non-transitory computer-readable medium comprising software for video processing, which when executed by a computer system, cause the computer system to perform operations comprising a method of:

receiving a video sequence of a field of view within an environment;

detecting targets in the video sequence;

receiving target geo-positional information transmitted by a geo-positional sensor; and

determining, using at least one processor of the computer system, correspondences between the targets detected in the video sequence and the target geo-positional information,

wherein determining the correspondences comprises:

determining corresponding pairs of targets detected in the video sequence and received target geo-positional information based on a predetermined criteria;

determining a probability density function of joint distribution of spatial positions of each determined corresponding pair; and

estimating high joint density modes.

2. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

receiving geo-positional information regarding a video camera that generated the video sequence, wherein the correspondences are further determined based on the geo-positional information of the video camera that generated the video sequence.

3. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

receiving geo-positional target information from a global positioning system (GPS) transmitter.

4. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

receiving geo-positional target information from an automatic identification system (AIS) transmitter.

5. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

receiving geo-positional target information from a blue force tracker (BFT) transmitter.

6. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

receiving geo-positional target information from a cell phone.

7. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

receiving geo-positional target information from a radio frequency identification (RFID) tag.

8. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

estimating at least one intrinsic parameter or at least one extrinsic parameter of a video camera that generated the video sequence based on the determined correspondences.

9. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

geo-registering the field of view of a video camera that generated the video sequence based at least on the determined correspondences.

10. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

geo-registering the field of view of a video camera that generated the video sequence based on geographic imaging system imagery of the environment.

11. The non-transitory computer-readable medium as set forth in claim 10 , wherein the geographic imaging system imagery comprises satellite imagery.

12. The non-transitory computer-readable medium as set forth in claim 10 , wherein the geographic imaging system imagery comprises nautical charts.

13. The non-transitory computer-readable medium as set forth in claim 10 , wherein the geographic imaging system imagery comprises electronic nautical charts.

14. The non-transitory computer-readable medium as set forth in claim 10 , wherein the geographic imaging system imagery comprises Digital Elevation Maps.

15. The non-transitory computer-readable medium as set forth in claim 10 , wherein the geographic imaging system imagery comprises a geospatial information system input.

16. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

determining a measurement of at least one of the target or the environment based on the determined correspondences.

17. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

weighting the density modes with correlation coefficients;

selecting a predetermined number of higher weighted density modes; and

determining homography for the selected higher weighted density modes.

18. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

based on the determined correspondences, at least one of:

detecting a target,

identifying a target,

classifying a target,

verifying a target, or

tracking a target.

19. The non-transitory computer-readable medium as set forth in claim 1 , wherein the method further comprises:

comparing the target geo-positional information to meta information of a corresponding target detected in the video sequence to obtain a comparison result; and

at least one of identifying or verifying at least one of the target detected in the video sequence or target geo-positional information based on the comparison result.

20. The method as set forth in claim 1 , further comprising:

weighting the density modes with correlation coefficients;

selecting a predetermined number of higher weighted modes; and

determining homography for the selected higher weighted modes.

21. A computer-based system to perform a method for video processing, the method comprising:

receiving a video sequence of a field of view within an environment;

detecting targets in the video sequence;

receiving target geo-positional information transmitted by a geo-positional sensor; and

determining, using at least one processor of the computer-based system, correspondences between the targets detected in the video sequence and the target geo-positional information,

wherein determining the correspondences comprises:

determining corresponding pairs of targets detected in the video sequence and received target geo-positional information based on a predetermined criteria;

determining a probability density function of joint distribution of spatial positions of each determined corresponding pair; and

estimating high joint density modes.

22. The system as set forth in claim 21 , wherein the method further comprises:

receiving geo-positional information regarding a video camera that generated the video sequence, wherein the correspondences are further determined based on the geo-positional information of the video camera that generated the video sequence.

23. The system as set forth in claim 21 , wherein the method further comprises:

estimating at least one intrinsic, or at least one extrinsic parameter of a video camera that generated the video sequence based on the determined correspondences.

24. The system as set forth in claim 21 , wherein the method further comprises:

geo-registering the field of view of a video camera that generated the video sequence based at least on the determined correspondences.

25. The system as set forth in claim 21 , further comprising:

geo-registering the field of view of a video camera that generated the video sequence based on geographic imaging system imagery of the environment.

26. The system as set forth in claim 25 , wherein the geographic imaging system imagery comprises satellite imagery.

27. The system as set forth in claim 21 , wherein the method further comprises:

determining measurements of at least one of the target or the environment based on the determined correspondences.

28. The system as set forth in claim 21 , wherein the method further comprises:

weighting the density modes with correlation coefficients;

selecting a predetermined number of higher weighted density modes; and

determining homography for the selected higher weighted density modes.

29. The system as set forth in claim 21 , wherein method further comprises:

based on the determined the determined correspondences, at least one of:

detecting a target,

identifying a target,

classifying a target,

verifying a target, or

tracking a target.

30. The system as set forth in claim 21 , wherein the method further comprises:

comparing the target geo-positional information to meta information of a corresponding target detected in the video sequence to obtain a comparison result; and

at least one of identifying or verifying at least one of the target detected in the video sequence or target geo-positional information based on the comparison result.

31. A method for video processing, comprising:

receiving a video sequence of a field of view within an environment;

detecting targets in the video sequence;

receiving target geo-positional information transmitted by a geo-positional sensor; and

determining, using at least one processor, correspondences between the targets detected in the video sequence and the target geo-positional information,

wherein determining the correspondences comprises:

determining corresponding pairs of targets detected in the video sequence and received target geo-positional information based on a predetermined criteria;

determining a probability density function of joint distribution of spatial positions of each determined corresponding pair; and

estimating high joint density modes.

32. The method as set forth in claim 31 , further comprising:

receiving geo-positional information regarding a video camera that generated the video sequence, wherein the correspondences are further determined based on the geo-positional information of the video camera that generated the video sequence.

33. The method as set forth in claim 31 , further comprising:

estimating at least one intrinsic parameter, or at least one extrinsic parameter of a video camera that generated the video sequence based on the determined correspondences.

34. The method as set forth in claim 31 , further comprising:

geo-registering the field of view of a video camera that generated the video sequence based at least on the determined correspondences.

35. The method as set forth in claim 31 , further comprising:

geo-registering the field of view of a video camera that generated the video sequence based on geographic imaging system imagery of the environment.

36. The method as set forth in claim 35 , wherein the geographic imaging system imagery comprises satellite imagery.

37. The method as set forth in claim 31 , further comprising:

determining a measurement of at least one of the target or the environment based on the determined correspondences.

38. The method as set forth in claim 31 , further comprising:

based on the determined the determined correspondences, at least one of:

detecting a target,

identifying a target,

classifying a target,

verifying a target, or

tracking a target.

39. The method as set forth in claim 31 , further comprising:

comparing the target geo-positional information and meta information of a corresponding target detected in the video sequence to obtain a comparison result; and

at least one of identifying or verifying at least one of the target detected in the video sequence or target geo-positional information based on the comparison result.

40. An apparatus to perform the video processing method of claim 31 .

41. A system to perform video processing of a video sequence generated by at least one video camera having a field of view within an environment, comprising:

a correspondence module to determine, using application specific hardware or at least one processor, correspondences between targets detected in the video sequence and corresponding target geo-positional information transmitted by a geo-positional sensor; and

at least one of:

a calibration and geo-registration module to determine a relationship between the video camera and geo-positional sensor and geo-register the video camera based at least on the determined correspondences; or

a metric rules and video understanding module to obtain measurement information regarding at least one of the environment or target detected in the video sequence,

wherein determining the correspondences comprises:

determining corresponding pairs of targets detected in the video sequence and received target geo-positional information based on a predetermined criteria;

determining a probability density function of joint distribution of spatial positions of each determined corresponding pair; and

estimating high joint density modes.

42. The system as set forth in claim 41 , comprising both of:

the calibration and geo-registration module to determine the relationship between the video camera and geo-positional sensor and geo-register the video camera based at least on the determined correspondences; and

the metric rules and video understanding module to obtain measurement information regarding at least one of the environment or target detected in the video sequence.