System and method of three-dimensional image capture and modeling
View Patent ↗System and method for constructing a 3D model of an object based on a series of silhouette and texture map images. In the exemplary embodiment an object is placed on a rotating turntable and a camera, which is stationary, captures images of the object as it rotates on the turntable. In one pass, the system captures a number of photographic images that will be processed into image silhouettes. In a second pass, the system gathers texture data. After a calibration procedure (used to determine the camera's focal length and the turntable's axis of rotation), a silhouette processing module determines a set of two-dimensional polygon shapes (silhouette contour polygons) that describe the contours of the object. The system uses the silhouette contour polygons to create a 3D polygonal mesh model of the object. The system determines the shape of the 3D model analytically by finding the areas of intersection between the edges of the model faces and the edges of the silhouette contour polygons. The system creates an initial, (rough) model of the 3D object from one of the silhouette contour polygons, then executes an overlaying procedure to process each of the remaining silhouette contour polygons. In the overlaying process, the system processes the silhouette contour polygons collected from each silhouette image, projecting each face of the (rough) 3D model onto the image plane of the silhouette contour polygons. The overlaying of each face of the (rough) 3D model onto the 2D plane of the silhouette contour polygons enables the present invention to determine those areas that are extraneous and should be removed from the (rough) 3D model. As the system processes the silhouette contour polygons in each image it removes the extraneous spaces from the initial object model and creates new faces to patch “holes.” The polygonal mesh model, once completed, can be transformed into a triangulated mesh model. In a subsequent step, the system uses a deterministic procedure to map texture from the texture images onto the triangles of the 3D mesh model, locating that area in the various texture map images that is “best” for each mesh triangle.
1. A method for processing two-dimensional images when creating a three-dimensional model of an object, the method comprising the steps of:
a. capturing a plurality of images of the object under lighting conditions in which the image is backlit such that the background in the image is lit and the object is not lit, the images comprising a set of pixel data with each pixel containing a light intensity value which describes the image;
b. capturing an image of the background under the backlighting conditions where the object has been removed before capturing the image, the image comprising a set pixel data with each pixel containing a light intensity value which describes the image;
c. examining each of the light intensity values of each pixel in the image of the background and building a first histogram which groups light intensity values within the image of the background into distribution categories;
d. creating a first mask to identify those pixels within the image of the background which fall into the distribution category that has the most pixels, the first mask keeping unmasked those pixels which fall into that distribution category and keeping masked those pixels which do not fall into that distribution category;
e. selecting one image from the plurality of images of the object;
f. examining each of the light intensity values of each pixel in the selected image of the object and building a second histogram which groups light intensity values within the selected image of the object into distribution categories;
g. creating a second mask to identify those pixels within the image of the object which fall into the distribution category of those pixels that have the lowest light intensity values, the second mask keeping unmasked those pixels which fall into that distribution category and keeping masked those pixels which do not fall into that distribution category;
h. matching the unmasked pixels in the first mask to corresponding unmasked pixels in the second mask and where corresponding matches can be found using the light intensity values of each such pixel pair to solve for a scaling coefficient and an offset coefficient; and
i. using the coefficients to process the remaining images of the object and identifying within those images those pixels which describe the object.
2. A computer-based system for processing two-dimensional images when creating a three-dimensional model of an object, the system comprising:
a. an image capturing component for capturing a plurality of images of the object under lighting conditions in which the image is backlit such that the background in the image is lit and the object is not lit, the images comprising a set of pixel data with each pixel containing a light intensity value which describes the image;
b. a second image capturing component for capturing an image of the background under the backlighting conditions where the object has been removed before capturing the image, the image comprising a set pixel data with each pixel containing a light intensity value which describes the image;
c. a computer processing module for examining each of the light intensity values of each pixel in the image of the background and building a first histogram which groups light intensity values within the image of the background into distribution categories;
d. a computer processing module for creating a first mask to identify those pixels within the image of the background which fall into the distribution category that has the most pixels, the first mask keeping unmasked those pixels which fall into that distribution category and keeping masked those pixels which do not fall into that distribution category;
e. a computer processing module for selecting one image from the plurality of images of the object;
f. a computer processing module for examining each of the light intensity values of each pixel in the selected image of the object and building a second histogram which groups light intensity values within the selected image of the object into distribution categories;
g. a computer processing module for creating a second mask to identify those pixels within the image of the object which fall into the distribution category of those pixels that have the lowest light intensity values, the second mask keeping unmasked those pixels which fall into that distribution category and keeping masked those pixels which do not fall into that distribution category;
h. a computer processing module for matching the unmasked pixels in the first mask to corresponding unmasked pixels in the second mask and where corresponding matches can be found using the light intensity values of each such pixel pair to solve for a scaling coefficient and an offset coefficient; and
i. a computer processing module for using the coefficients to process the remaining image of the object and identifying within those images those pixels which describe the object.