System and method for robotic camera calibration
A processor and camera capture data corresponding to a position of a ball and automatically adjust orientation and zoom level of the camera. The processor pre-calibrates the camera so an initial orientation is known in a world coordinate system (WCS) and so different zoom levels are associated with respective parameter values. A first position of the ball in a first image is detected and a first zoom level of the first image and intrinsic parameter values for the first zoom level are read. Based on pan and tilt relative to the initial orientation, the first orientation of the camera is determined. A 3D line through the camera and the ball in WCS is determined based on the first position, the first orientation and parameter values for the first zoom level and a 3D position is determined in WCS along the line based on information extrinsic to the camera.
1 . A system, comprising:
a robotic camera configured to capture data corresponding to a position of a sports ball, the robotic camera being further configured to automatically adjust an orientation and a zoom level of the robotic camera in response to the captured data or an external command; and
a processing arrangement coupled to the robotic camera configured to:
pre-calibrate the robotic camera so that an initial orientation of the robotic camera is known in a world coordinate system and associate each of a plurality of different zoom levels used by the robotic camera with a respective intrinsic parameter value;
detect a first image position of the sports ball on a surface of a sports play area in an image coordinate system in a first image, the first image being captured with the robotic camera in a first orientation;
read from the robotic camera a first zoom level associated with the first image and a first intrinsic parameter value associated with the first zoom level;
determine the first orientation of the robotic camera based on a pan and a tilt of the robotic camera relative to the initial orientation;
determine a three-dimensional line extending through the robotic camera and the sports ball in the world coordinate system based on the detected first image position of the sports ball, the determined first orientation and the first intrinsic parameter value for the first zoom level; and
determine a three-dimensional position of the sports ball in the world coordinate system located along the three-dimensional line based on information extrinsic to the robotic camera,
wherein the information extrinsic to the robotic camera includes information received from a three-dimensional model of at least a portion of the surface of the sports play area that encompasses the three-dimensional position of the sports ball, the processing arrangement determining the three-dimensional position of the sports ball by locating an intersection of the three-dimensional line and the three-dimensional model.
2 . The system of claim 1 , further comprising:
a pan and tilt sensor fixed to the robotic camera,
wherein the processing arrangement is further configured to read the pan and the tilt of
the robotic camera relative to the initial orientation from the pan and tilt sensor.
3 . The system of claim 1 , wherein the pan and the tilt of the robotic camera is determined from a control signal used to adjust the robotic camera to the first orientation, wherein the control signal is generated in response to tracking data for the sports ball captured prior to capture of the first image.
4 . The system of claim 1 , wherein the robotic camera is further configured to automatically adjust a crop in response to the captured data or the external command.
5 . The system of claim 1 , wherein the intrinsic parameters comprise a focal length, a principal point, and a lens distortion.
6 . The system of claim 1 , wherein the information extrinsic to the robotic camera includes information from a sensor relating to a distance of the sports ball from the robotic camera.
7 . A method, comprising:
pre-calibrating a robotic camera so that an initial orientation of the robotic camera is known in a world coordinate system and associating each of a plurality of zoom levels used by the robotic camera with a respective intrinsic parameter value, wherein the robotic camera is configured to capture data corresponding to a position of a sports ball, the robotic camera being further configured to automatically adjust an orientation and zoom level in response to the captured data or an external command;
detecting a first image position of the sports ball on a surface of a sports play area in an image coordinate system in a first image, the first image being captured with the robotic camera in a first orientation;
reading a first zoom level from the robotic camera and a first intrinsic parameter value for the first zoom level of the robotic camera corresponding to the first image;
determining the first orientation of the robotic camera based on a pan and a tilt of the robotic camera relative to the initial orientation;
determining a three-dimensional line extending through the robotic camera and the sports ball in the world coordinate system based on the detected first image position of the sports ball, the determined first orientation and the first intrinsic parameter value for the first zoom level; and
determining a three-dimensional position of the sports ball in the world coordinate system located along the three-dimensional line based on information extrinsic to the robotic camera,
wherein the information extrinsic to the robotic camera includes information received from a three-dimensional model of at least a portion of the surface of the sports play area that encompasses the three-dimensional position of the sports ball, the three-dimensional position of the sports ball being determined by locating an intersection of the three-dimensional line and the three-dimensional model.
8 . A processor coupled to a robotic camera configured to perform operations comprising:
pre-calibrating the robotic camera so that an initial orientation of the robotic camera is known in a world coordinate system and each different zoom level used by the robotic camera is associated with a respective intrinsic parameter value, wherein the robotic camera is configured to capture data corresponding to a position of a sports ball, the robotic camera being further configured to automatically adjust an orientation and zoom level in response to the captured data or an external command;
detecting a first image position of the sports ball on a surface of a sports play area in an image coordinate system in a first image, the first image being captured with the robotic camera in a first orientation;
reading a first zoom level from the robotic camera and a first intrinsic parameter value for the first zoom level corresponding to the first image;
determining the first orientation of the robotic camera based on a pan and a tilt of the robotic camera relative to the initial orientation;
determining a three-dimensional line extending through the robotic camera and the sports ball in the world coordinate system based on the detected first image position of the sports ball, the determined first orientation and the first intrinsic parameter value for the first zoom level; and
determining a three-dimensional position of the sports ball in the world coordinate system located along the three-dimensional line based on information extrinsic to the robotic camera,
wherein the information extrinsic to the robotic camera includes information received from a three-dimensional model of at least a portion of the surface of the sports play area that encompasses the three-dimensional position of the sports ball, the three-dimensional position of the sports ball being determined by locating an intersection of the three-dimensional line and the three-dimensional model.