Organic camera-pose mapping
A system and method of organically generating a camera-pose map is disclosed. A target image is obtained of a location deemed suitable for augmenting with a virtual augmentation or graphic. An initial camera-pose map is created having a limited number of calibrated camera-pose images having calculated camera-pose locations and homographies to the target image. Then, during the event, the system automatically obtains current images of the event venue and determines homographies to the nearest calibration camera-pose image in the camera-pose map. The separation in camera-pose space between the current images and the camera-pose images are calculated. If this separation is less than a predetermined threshold, that current image is fully calibrated and added to the camera-pose map, thereby growing the map organically.
1. A method of organically generating a camera-pose map, comprising:
obtaining a target image of an event venue taken immediately prior to a commencement of an event occurring at said event venue using a fixed camera located at said event venue, said target image containing a target location designated as a location for the insertion of a virtual graphic;
designating a camera-pose of said target image to be an origin of a camera-pose space;
obtaining at least one calibrated camera-pose image of said event venue, by:
taking, immediately prior to said commencement of said event, an image of said event venue using said fixed camera, said image having a camera pose that differs from that of said target image;
determining an augmentation homography of said calibrated camera pose image;
determining a camera-pose location of said image using said augmentation homography, said camera-pose location being a position of said image in camera pose location; and,
designating said image as a calibrated camera-pose image;
obtaining a current image of said event venue during said event using said fixed camera;
determining a current-to-calibrated homography of said current image to one of said calibrated camera-pose images having a camera-pose location closest to an estimated camera-pose location of said current image; and,
calculating a camera-pose location of said current image using said current-to-calibrated homography and said augmentation homography of said calibrated camera-pose image;
calculating a distance in camera-pose space between said current image and said one of said camera-pose images; and,
when said distance in camera-pose space is less than a predetermined threshold, designating said current image as a calibrated camera-pose image thereby enlarging said camera-pose map of said fixed camera at said event.
2. The method of claim 1 , wherein, said estimated camera-pose location of said current image is a camera pose location of an immediately preceding current image.
3. The method of claim 1 , wherein, distance in camera-pose space between said current image and said one of said calibrated camera-pose images is calculated using the equation:
D =Sqrt(( P 2− P 1)2+( Z 2− Z 1)2+( T 2− T 1)2),
where D represents the distance, P2 represents the pan component of the current image in camera-pose space, P1 represents the pan component of the calibrated camera-pose image in camera-pose space, Z2 represents the zoom component of the current image in camera-pose space, Z1 represents the zoom component of the calibrated camera-pose image in camera-pose space, and, T2 represents the translation component of the current image in camera-pose space, and T1 represents the translation component of the calibrated camera-pose image in camera-pose space.
4. The method of claim 1 , wherein, said homography of said current image to one of said calibrated camera-pose images is calculated using a direct linear transformation.
5. The method of claim 4 , wherein, said direct linear transformation is solved using a singular value decomposition.
6. The method of claim 1 , wherein, said predetermined threshold is 1/100 of a distance between a camera-pose image when the camera is panned to an extreme right pan position with medium zoom, and a camera-pose image when the camera is panned to an extreme left pan position with medium zoom.
7. A system of organically generating a camera-pose map, comprising:
a digital video camera located at a fixed location at an event venue;
a programmable digital computer functionally connected to said digital video camera;
designating, by an operator, a target location, said target location being an area deemed suitable for augmenting with a virtual augmentation;
obtaining, by said operator using said computer and said camera, a target image of said target location within said event venue immediately prior to a commencement of an event at said event venue;
designating a camera-pose of said target image to be an origin of a camera-pose space;
obtaining and calibrating, by said operator using said computer and said camera, at least one calibrated camera-pose image, said calibrated camera-pose image having a calculated camera-pose location in said camera-pose space and a calculated augmentation homography to said target image; and,
said computer being programmed to automatically preform the functions of:
obtaining, during an event at said event venue, a current image of said event venue using said fixed camera;
determining a current-to-calibrated homography of said current image to one of said calibrated camera-pose images having a camera-pose location closest to an estimated camera-pose location of said current image;
calculating a camera-pose location of said current image using said current-to-calibrated homography and said augmentation homography of said calibrated camera pose image;
calculating a distance in camera-pose space between said current image and said one of said camera-pose images; and,
when said distance in camera-pose space is less than a predetermined threshold, designating said current image as a calibrated camera-pose image.
8. The system of claim 7 , wherein, said estimated camera-pose location of said current image is a camera pose location of an immediately preceding current image.
9. The system of claim 7 , wherein, distance in camera-pose space between said current image and said one of said camera-pose images is calculated using the equation:
D =Sqrt(( P 2− P 1)2+( Z 2− Z 1) 2 +( T 2− T 1)2),
where D represents the distance, P2 represents the pan component of the current image in camera-pose space, P1 represents the pan component of the calibrated camera-pose image in camera pose space, Z2 represents the zoom component of the current image in camera-pose space, Z1 represents the zoom component of the calibrated camera-pose image in camera-pose space, T2 represents the translation component of the current image in camera-pose space, and T1 represents the translation component of the calibrated camera-pose image in camera-pose space.
10. The system of claim 7 , wherein, said current-to-calibrated homography of said current image to one of said calibrated camera-pose images is calculated using a direct linear transformation.
11. The system of claim 10 , wherein, said direct linear transformation is solved using a singular value decomposition.
12. The system of claim 7 , wherein, said predetermined threshold is 1/100 of a distance between a camera-pose image when the camera is panned to an extreme right pan position with medium zoom, and a camera-pose image when the camera is panned to an extreme left pan position with medium zoom.