Multi view camera registration
A system for registering one or more cameras and/or creating an accurate three-dimensional (3D) model of a world space environment including back projecting at least one image from at least one of a plurality of camera views to the 3D model based on a set of existing camera parameters. The back projected image is added as a texture for the 3D model. This texture is automatically compared to one or more images from other camera views using a color space comparison of images to determine a set of differences or errors. The camera parameters and the 3D model are automatically adjusted to minimized the differences or errors. Over time, the parameters and the 3D model converge on a state that can be used to track moving objects, insert virtual graphics and/or perform other functions.
1 . A method for imposing virtual graphics into camera video, comprising:
constructing an initial approximate 3D model of a space based on known information regarding the space, wherein the initial approximate 3D model is flat;
generating initial estimates for camera parameters for a plurality of cameras;
receiving camera images from the plurality of cameras and automatically updating the initial approximate 3D model and the camera parameters based on the camera images, wherein the camera images include identified points, lines, or conics in the space;
determining world coordinates for at least one virtual graphic to impose onto video produced by one or more of the plurality of cameras;
transforming the world coordinates for the at least one virtual graphic to camera coordinates, and imposing the at least one virtual graphic on the video produced by the one or more of the plurality of cameras; and
collecting data from at least one extrinsic sensor mounted on the one or more of the plurality of cameras;
wherein the at least one extrinsic sensor comprises a pan sensor and/or a tilt sensor.
2 . The method of claim 1 , wherein automatically updating the initial approximate 3D model and camera parameters includes comparing back projected images from a first of the plurality of cameras to back projected images from a second of the plurality of cameras.
3 . The method of claim 2 , further comprising determining a set of error metrics by comparing the back projected images from the first of the plurality of cameras to back projected images from the second of the plurality of cameras, and aggregating the set of error metrics into a combined differences value.
4 . The method of claim 3 , wherein automatically updating the initial approximate 3D model includes repeating the back projecting of the images from the plurality of cameras and comparing the back projected images until the combined differences value is less than a threshold value.
5 . The method of claim 3 , wherein comparing the back projected images from the first of the plurality of cameras to the back projected images from the second of the plurality of cameras includes comparing color values of the back projected images from the first of the plurality of cameras to color values of the back projected images from the second of the plurality of cameras.
6 . The method of claim 1 , further comprising tracking at least one moving object in the video produced by the one or more of the plurality of cameras.
7 . The method of claim 1 , wherein automatically updating the initial approximate 3D model includes modifying geometries and/or textures of the initial approximate 3D model.
8 . The method of claim 1 , wherein the camera parameters include a camera location, distortion coefficients, vertical field of view, tilt angle, pan angle, roll angle, and/or zoom level.
9 . An apparatus for imposing virtual graphics into camera video, comprising:
one or more processors; and
one or more non-transitory, tangible computer readable storage media connected to the one or more processors;
wherein the one or more processors are configured to construct an initial approximate 3D model of a space based on known information regarding the space, wherein the initial approximate 3D model is flat;
wherein the one or more processors are configured to generate initial estimates for camera parameters for a plurality of cameras;
wherein the one or more processors are configured to receive camera images from the plurality of cameras and automatically update the initial approximate 3D model and the camera parameters based on the camera images, wherein the camera images include identified points, lines, or conics in the space;
wherein the one or more processors are configured to determine world coordinates for at least one virtual graphic to impose onto video produced by one or more of the plurality of cameras;
wherein the one or more processors are configured to transform the world coordinates for the at least one virtual graphic to camera coordinates, and imposing the at least one virtual graphic on the video produced by the one or more of the plurality of cameras;
wherein the one or more of the plurality of cameras includes at least one extrinsic sensor comprising a pan sensor and/or a tilt sensor; and
wherein the plurality of cameras are operable to capture ultraviolet data.
10 . The apparatus of claim 9 , wherein the one or more processors automatically updating the initial approximate 3D model and camera parameters includes comparing images from a first of the plurality of cameras to images from a second of the plurality of cameras.
11 . The apparatus of claim 10 , where the one or more processors are configured to determine a set of error metrics by comparing the images from the first of the plurality of cameras to the images from the second of the plurality of cameras, and aggregating the set of error metrics into a combined differences value.
12 . The apparatus of claim 11 , wherein the one or more processors automatically updating the initial approximate 3D model includes repeatedly comparing the images from the plurality of cameras until the combined differences value is less than a threshold value.
13 . The apparatus of claim 11 , wherein comparing the images from the first of the plurality of cameras to the images from the second of the plurality of cameras includes comparing color values of the images from the first of the plurality of cameras to color values of the images from the second of the plurality of cameras.
14 . The apparatus of claim 9 , wherein the one or more processors are further operable to track at least one moving object in the video produced by the one or more of the plurality of cameras.
15 . The apparatus of claim 9 , wherein the one or more processors automatically updating the initial approximate 3D model includes modifying geometries and/or textures of the initial approximate 3D model.
16 . The apparatus of claim 9 , wherein the camera parameters include a camera location, distortion coefficients, vertical field of view, tilt angle, pan angle, roll angle, and/or zoom level.
17 . An apparatus for imposing virtual graphics into camera video, comprising:
one or more processors; and
one or more non-transitory, tangible computer readable storage media connected to the one or more processors;
wherein the one or more processors are configured to construct an initial approximate 3D model of a space based on known information regarding the space, wherein the initial approximate 3D model is flat;
wherein the one or more processors are configured to generate initial estimates for camera parameters for a plurality of cameras;
wherein the one or more processors are configured to receive camera images from the plurality of cameras and automatically update the initial approximate 3D model and the camera parameters based on the camera images by comparing images from a first of the plurality of cameras to images from a second of the plurality of cameras, wherein the camera images include identified points, lines, or conics in the space;
where the one or more processors are configured to determine a set of error metrics by comparing the images from the first of the plurality of cameras to the images from the second of the plurality of cameras, and aggregating the set of error metrics into a combined differences value;
wherein the one or more processors are configured to impose the at least one virtual graphic onto video produced by one or more of the plurality of cameras;
wherein the one or more of the plurality of cameras includes at least one extrinsic sensor comprising a pan sensor and/or a tilt sensor; and
wherein the one or more processors are operable to perform multi-view camera registration.
18 . The apparatus of claim 17 , wherein the one or more processors automatically updating the initial approximate 3D model includes repeatedly comparing the images from the plurality of cameras until the combined differences value is less than a threshold value.
19 . The apparatus of claim 17 , wherein comparing the images from the first of the plurality of cameras to the images from the second of the plurality of cameras includes comparing color values of the images from the first of the plurality of cameras to color values of the images from the second of the plurality of cameras.
20 . The apparatus of claim 17 , wherein the one or more processors are further operable to track at least one moving object in the video produced by the one or more of the plurality of cameras.