Methods for collecting and processing image information to produce digital assets
Paired images of substantially the same scene are captured with the same freestanding sensor. The paired images include reflected light illuminated with controlled polarization states that are different between the paired images. Information from the images is applied to a convolutional neural network (CNN) configured to derive a spatially varying bi-directional reflectance distribution function (SVBRDF) for objects in the paired images. Alternatively, the sensor is fixed and oriented to capture images of an object of interest in the scene while a light source traverses a path that intersects the sensor's field of view. Information from the paired images of the scene and from the images captured of the object of interest when the light source traverses the field of view are applied to a CNN to derive a SVBDRF for the object of interest. The image information and the SVBRDF are used to render a representation with artificial lighting conditions.
1. A method for processing image information, the method comprising:
receiving a first data set representative of a cross-polarized image of a scene, wherein subject matter is illuminated to substantially avoid shadows in the cross-polarized image to provide an albedo surface texture of objects in a field of view;
receiving a second data set representative of a co-polarized image of substantially the same scene illuminated to substantially avoid shadows in the co-polarized image;
receiving a third data set representative of images of substantially the same scene wherein subject matter including at least one light probe in the field of view is illuminated with a repositioned light source between a first location and a second location such that some portion of a first line defined by the first location and the second location intersects the field of view, the images in the third data set sampling reflections cast from the at least one light probe and subject matter with surfaces of interest as the light source traverses a path; and
using the first and second data sets in combination with the third data as inputs to a convolutional neural network configured to derive a spatially varying bidirectional reflectance distribution function (SVBRDF) representative of subject matter in the scene to produce a refined diffuse albedo surface texture and a specular roughness surface texture.
2. The method of claim 1 , wherein the number of images captured for a respective number of positions the light source translates between a first location and a second location is increased as light in the field of view is reflected by relatively complex materials in the scene.
3. The method of claim 1 , wherein the first and the second data sets are captured with an image sensor coupled to a freestanding chassis.
4. The method of claim 1 , wherein light reflected by the at least one light probe is used to determine a location of the light source.
5. The method of claim 4 , wherein the stationary image sensor is located and oriented to capture images of a substantially planar surface present in the field of view, the normal of the planar surface substantially facing the image sensor.
6. The method of claim 1 , wherein the first data set and the second data set include subject matter illuminated with a controlled illumination source arranged about a perimeter of an image sensor.
7. The method of claim 1 , wherein the third data set representative of images of substantially the same scene further includes images captured with the light source translating between one of the first location or the second location to a third location such that a second line traversed by the light source is substantially orthogonal to the first line traversed by the light source.
8. The method of claim 7 , wherein the third data set representative of images of substantially the same scene further includes images captured with the light source translating between one of the first, second or third locations and a fourth location such that a third line traversed by the light source is substantially orthogonal to both the first line and the second line traversed by the light source.
9. The method of claim 1 , wherein the light source translating between the first location and the second location traverses a path other than a straight line.
10. The method of claim 1 , wherein the first and second data sets respectively include a set of exposures captured from more than one perspective of a scene, and wherein the third data set includes a set of exposures captured from a single perspective of a scene that is substantially shared with at least one member of the first data set and at least one member of the second data set.
11. The method of claim 1 , further comprising:
using the first and second data sets, the spatially varying bidirectional reflectance distribution function, and a sensor orientation to generate a UV map; and
receiving information representative of a surface geometry of the subject matter, wherein using the first and second data sets further includes applying color information from the data set over the surface geometry.
12. The method of claim 11 , further comprising:
generating a virtual environment from the surface geometry of the subject matter and the UV map;
receiving a preferred orientation; and
modifying the virtual environment in response to the preferred orientation.
13. The method of claim 12 , further comprising:
receiving information characterizing a virtual light source; and
modifying the virtual environment in response to the virtual light source and the preferred orientation.
14. The method of claim 13 , wherein information characterizing the virtual light source includes one or more of identifying a location, a luminous flux, and a frequency range.
15. The method of claim 13 , wherein the virtual environment is used in a product selected from the group consisting of an exhibit, video game, cinematic production, and a teaching aid.