Automatic calibration of PTZ camera system

1. A method of calibrating of a pan-tilt-zoom (PTZ) camera system comprising a PTZ camera, the method comprising performing the following steps:

determining a radial distortion of the PTZ camera;

determining a base focal length of the PTZ camera;

determining a zoom and magnification profile of the PTZ camera;

determining an actuation delay of the PTZ camera system;

determining a pan position mode speed and a tilt position mode speed of the PTZ camera;

determining a pan velocity profile and a tilt velocity profile of the PTZ camera;

determining a zoom position profile and a zoom duration profile of the PTZ camera;

determining a query delay of the PTZ camera system; and

determining a minimum query interval of the PTZ camera system.

2. The method of claim 1 , wherein determining the radial distortion of the PTZ camera comprises:

generating a spherical mosaic;

identifying sets of feature points from the spherical mosaic;

determining a range of reasonable radial distortion coefficients;

warping one or more sets of feature points according to the determined range of reasonable radial distortion coefficients;

determining a planar homography for one or more sets of warped feature point;

determining a reprojection error for one or more planar homographies; and

determining the radial distortion coefficient for one or more sets of feature points.

3. The method of claim 2 , wherein the radial distortion coefficient for one or more sets of feature points is characterized by the equation

k

^

=

arg

⁢

⁢

min

k

⁢

∑

s

⁢

∑

i

⁢



x

u

⁡

(

x

s

,

i

,

k

)

-

H

s

⁢

x

u

⁡

(

x

s

,

i

′

,

k

)



2

where k is the radial distortion coefficient, x u is the ideal projection of a point an image plane, s is a pair of feature points, x s,i and x′ s,i represent the i th corresponding feature point pair in set s, and H S is the planar homography.

4. The method of claim 1 , wherein determining the base focal length of the PTZ camera comprises:

generating a spherical mosaic with a feature track; and

determining a base focal length by analyzing a conic trajectory created by the feature track on an image plane.

5. The method of claim 4 , wherein the base focal length is characterized by the equation

(

f

,

θ

,

ψ

)

=

arg

⁢

⁢

min

f

,

θ

,

ψ

⁢

∑

j

⁢

e

^

j

2

⁡

(

x

i

,

j

,

f

,

θ

,

ψ

)

where f is the base focal length, θ is a tilt, ψ is a roll, x i,j is a set of image plane measurements on the conic for the j th feature, and ê 2 is a minimum residual error.

6. The method of claim 1 , wherein determining the zoom and magnification profile of the PTZ camera comprises:

executing a plurality of zoom commands with the PTZ camera, wherein a first zoom command of the plurality of zoom commands sets the PTZ camera to a starting zoom level;

capturing a plurality of images with the PTZ camera, wherein images in the plurality of images are captured after a zoom command from the plurality of zoom commands has been executed;

estimating a plurality of image magnifications based on images from the plurality of images; and

determining a zoom and magnification profile of the PTZ camera based on the plurality of image magnifications and the plurality of zoom commands.

7. The method of claim 6 , wherein estimating a plurality of image magnifications comprises comparing a first image from the plurality of images to a second image from the plurality of images, wherein the second image is captured subsequent to the first image.

8. The method of claim 7 , wherein comparing comprises analyzing images and determining magnification using computer-based image processing.

9. The method of claim 1 , wherein determining the actuation delay of the PTZ camera system comprises:

issuing a velocity command to the PTZ camera;

capturing successive frames from the PTZ camera;

detecting PTZ camera motion by comparing successive captured frames; and

determining an actuation delay of the PTZ camera, wherein the actuation delay is the amount of time between issuing the velocity command to the PTZ camera and observing PTZ camera motion.

10. The method of claim 1 , wherein determining the pan position mode speed and the tilt position mode speed of the PTZ camera comprises:

setting a speed for the PTZ camera;

issuing a pan command or a tilt command to the PTZ camera;

capturing successive frames from the PTZ camera;

detecting PTZ camera movement by comparing successive captured frames;

detecting the lack of PTZ camera movement by comparing captured successive frames; and

determining the pan position mode speed of the PTZ camera or the tilt position mode speed of the PTZ camera, wherein the pan position mode speed or the tilt position mode speed is the amount of time between detecting PTZ camera movement and detecting the lack of PTZ camera movement.

11. The method of claim 1 , wherein determining the pan velocity profile and the tilt velocity profile of the PTZ camera comprises:

setting a plurality of speeds for the PTZ camera;

setting the PTZ camera to a first position;

receiving a plurality of time intervals, wherein successive time intervals in the plurality of time intervals increase in length of time;

issuing a plurality of pan commands to the PTZ camera or a plurality of tilt commands to the PTZ camera executed during time intervals of the plurality of time intervals, wherein the PTZ camera returns to the first position after expiration of the time intervals;

determining a plurality of camera positions, wherein camera positions from the plurality of camera positions are determined after a pan command from the plurality of pan commands or a tilt command from the plurality of tilt commands has been issued and after a time interval from the plurality of time intervals;

determining a plurality of pan velocities or a plurality of tilt velocities based on the plurality of positions and the plurality of time intervals; and

determining a pan velocity profile of the PTZ camera or a tilt velocity profile of the PTZ camera, wherein the pan velocity profile is based on the plurality of pan velocities and the plurality of time intervals or the tilt velocity profile is based on the plurality of tilt velocities and the plurality of time intervals.

12. The method of claim 11 ,wherein determining the plurality of camera positions comprises:

performing a loop comprising:

setting the PTZ camera to the first position, and

issuing a pan command from the plurality of pan commands or a tilt command from the plurality of tilt commands to the PTZ camera during a time interval in the plurality of time intervals.

13. The method of claim 1 , wherein determining the zoom position profile and the zoom duration profile of the PTZ camera comprises:

setting a minimum zoom level for the PTZ camera;

setting a zoom in speed for the PTZ camera;

setting a plurality of zoom in periods for the PTZ camera;

executing a plurality of zoom in operations from the minimum zoom level, at the zoom in speed, during the plurality of zoom in periods;

determining a plurality of zoom levels for the PTZ camera based on a position of the camera after the zoom in operations; and

determining a zoom position profile or a zoom duration profile of the PTZ camera based on the plurality of zoom levels.

14. The method of claim 13 , wherein executing the plurality of zoom in operations comprises:

looping through the plurality of zoom in operations, wherein during a loop, the PTZ camera zooms in from the minimum zoom level, at the zoom in speed, during a zoom in period of the plurality of zoom in periods.

15. The method of claim 1 , wherein determining the query delay of the PTZ camera system comprises:

sending a query command to the PTZ camera;

receiving a response to the query command from the PTZ camera; and

determining the query delay of the PTZ camera, wherein the query delay is the amount of time between sending the query command and receiving the response to the query command.

16. The method of claim 1 , wherein determining the minimum query interval of the PTZ camera system comprises:

(a) sending a first query command to the PTZ camera;

(b) sending a second query command to the PTZ camera after the expiration of a wait interval;

(c) receiving a response to the second query command;

(d) determining whether the received response comprises a valid response to the second query command and setting the minimum query interval equal to the wait interval when the minimum query interval comprises the valid response to the second query command; and

(e) increasing the wait interval and repeating steps (a) through (e) when a non-valid response to the second query command is received.

17. The method of claim 1 , wherein determining the radial distortion of the PTZ camera comprises:

receiving successive video image frames from a PTZ camera;

constructing a spherical mosaic engine from the successive video images;

determining when the PTZ camera has completed a first scan based on the successive images;

estimating the center frame of the first scan;

centering the spherical mosaic based on the estimated center frame;

estimating a focal length of the PTZ camera;

detecting feature points based on the successive images;

reprojecting the spherical mosaic; and

determining a radial distortion of the PTZ camera based on the feature points and the spherical mosaic.

18. The method of claim 1 , wherein determining the base focal length of the PTZ camera comprises:

receiving successive video image frames from the PTZ camera;

constructing a spherical mosaic engine from the successive video images;

determining when the PTZ camera has completed a first scan based on the successive images;

estimating the center frame of the first scan;

centering the spherical mosaic based on the estimated center frame;

estimating a focal length of the PTZ camera;

detecting one or more feature points based on the successive images;

reprojecting the spherical mosaic; and determining the base focal length of the PTZ camera by analyzing a conic trajectory created by the one or more feature points in the spherical mosaic.

19. A pan-tilt-zoom (PTZ) camera calibration system comprising:

a PTZ camera; and

one or more computers to implement the following engines:

a radial distortion engine to determine a radial distortion of the PTZ camera;

a base focal length engine to determine a base focal length of the PTZ camera;

a zoom and magnification engine for determine a zoom and magnification profile of the PTZ camera;

an actuation delay engine to determine an actuation delay of the PTZ camera system;

a pan/tilt position mode speed engine to determine a pan position mode speed and a tilt position mode speed of the PTZ camera;

a pan/tilt velocity engine to determine a pan velocity profile and a tilt velocity profile of the PTZ camera;

a zoom position and duration engine to determine a zoom position profile and a zoom duration profile of the PTZ camera;

a query delay engine to determine a query delay of the PTZ camera system; and

a minimum query interval engine to determine a minimum query interval of the PTZ camera system.