System and method for projection mapping
A system and a process for accurately aligning projection mapping systems using the assistance of camera sensors. The calibration process involves first projecting a modulated gray code image sequence that allows the calibration cameras to identify a dense set of image correspondences between the calibration cameras and the projectors. The calibration cameras are then aligned to the screen surface through the selection of key points or fiducials on the screen surface that are then identified in the camera images. Once the camera(s) are aligned to the screen surface, the image correspondences between the projectors and cameras are used to determine the alignment of the projectors to the screen surface.
1 . A method for projection mapping, for a plurality of projectors, responsive to an image processor arrangement, in which the processor arrangement receives image data from projected images of each projector of the plurality of projectors, the method comprising:
projecting a sequential calibration pattern onto a projection surface;
capturing one or more images of the projection surface including at least one pattern of the sequential calibration pattern projected thereon;
computing a projector to camera transform;
computing a camera to projection surface transform as a mapping between camera pixels to 3D locations in a coordinate space of the projection surface based on a portion of a set of projection surface image correspondences from an image of the projection surface that correspond to one or more of a set of projection surface model correspondences from a digital model of the projection surface that is independent of the one or more captured images;
determining a projector to projection surface transform; and
determining a frustum for each projector.
2 . The method of claim 1 , wherein the sequential calibration pattern comprises a modulated gray code sequence.
3 . The method of claim 2 , wherein the modulated gray code sequence is generated by modulating a predetermined gray code sequence with a first pattern.
4 . The method of claim 1 , wherein the one or more images are captured with at least one camera.
5 . The method of claim 1 , wherein the projection surface comprises a three-dimensional (3D) projection surface.
6 . The method of claim 1 , wherein computing the projector to camera transform comprises: for each pattern of the sequential calibration pattern, determining whether a pixel intensity of displayed cells are the same or opposite as compared to a first pattern, reconstructing a gray code bit sequence at each camera pixel, and classifying each camera pixel to particular cells in the projector.
7 . The method of claim 6 , further comprising, computing a centroid of each cell relative to the camera.
8 . The method of claim 1 , wherein computing the camera to projection surface transform comprises: selecting a first plurality of correspondences in a model of the projection surface, and selecting a second plurality of correspondences in an image of the projection surface, wherein the second plurality of correspondences correspond with the first plurality of correspondences.
9 . The method of claim 1 , further comprising, manually adjusting the projector to projection surface transform using a grid-based adjustment tool, so as to provide a fine tuning to the projection mapping.
10 . The method of claim 1 , further comprising, computing a blend map, including generating a contour image with each projector onto the surface based upon the frustum for the projector.
11 . The method of claim 1 , further comprising, generating a distance transform image for each projector.
12 . The method of claim 11 , further comprising, generating a blend weight image for each projector.
13 . The method of claim 12 , further comprising, generating a final blend map for each projector.
14 . A system for projection mapping for a plurality of projectors, responsive to an image processor arrangement, in which the processor arrangement receiving image data with a camera from projected images of each projector of the plurality of projectors, the system comprising:
a projection process of the processing arrangement that is adapted to project a sequential calibration pattern onto a projection surface, at a predetermined image field thereof, with each projector, respectively;
the image data including one or more images, within the predetermined image field, with respect to each projector of each pattern of a plurality of patterns of the sequential calibration pattern, wherein the sequential calibration pattern comprises a modulated gray code sequence, which is generated by modulating a predetermined gray code sequence with a first pattern;
the image processor arrangement computing a camera to projection surface transform based on a portion of a set of first correspondences from an image of the projection surface that correspond to one or more of a set of second correspondences from a digital model of the projection surface that is independent of the received image data; and
the predetermined gray code sequence being adapted to maintain a substantially consistent average intensity across the image field in each pattern in the sequence.
15 . The system of claim 14 wherein the image processor arrangement is adapted to generate a distance transform image for each projector.
16 . The system of claim 15 wherein the image processor arrangement is adapted to generate a blend weight image for each projector.
17 . The system of claim 16 wherein the image processor arrangement is adapted to generate a final blend map for each projector.
18 . The system of claim 14 wherein initial patterns of the gray code sequence include a first pattern that is a tessellation of light and dark features, and at least one subsequent pattern in the gray code sequence that defines a bifurcated mirror image of the first pattern on each side of each of a vertical bifurcating center line and a horizontal bifurcating center line.
19 . The system of claim 14 wherein the image processor is constructed and arranged to compute a projector to camera transform.
20 . The system of claim 19 wherein the image processor is constructed and arranged to determine a projector to projection surface transform and a frustum for each projector.