IP Library Granted Patent US 11,210,839
Granted Patent B2
US 11,210,839 · App. 16/812,860 · Granted Dec 28, 2021

Photometric image processing

Inventors: Amirhessam Moltaji (Vancouver, CA); Kevin Loken (Surrey, CA); Ryan Hietanen (Vancouver, CA)
Assignee: Electronic Arts Inc.
G06T15/04G06T7/90G06T7/97G06T11/001G06T2207/10024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,210,839
App. No.
16/812,860
Granted
Dec 28, 2021
Kind
B2
Abstract

An example method of photometric image processing may comprise: receiving a plurality of images of a three-dimensional object, wherein the plurality of images has been acquired by a plurality of cameras using a plurality of illumination and polarization patterns; performing color calibration of the plurality of images to produce a plurality of color-calibrated images; generating, using the plurality of color-calibrated images, a polygonal mesh simulating geometry of the three-dimensional object; producing a plurality of partial UV maps by projecting the plurality of color-calibrated images onto the polygonal mesh; generating a plurality of masks, wherein each mask of the plurality of masks is associated with a camera of the plurality of cameras, wherein the mask defines a UV space region that is covered by a field of view of the camera; blending, using the plurality of masks, the plurality of partial UV maps; and generating one or more texture maps representing the three-dimensional object.

Claims (54)

1. A method, comprising:

receiving a plurality of input images of a three-dimensional object, wherein the plurality of input images has been acquired by a plurality of cameras using a plurality of illumination and polarization patterns;

producing a plurality of adjusted images by adjusting each input image of the plurality of input images by a pre-computed gain value, wherein the pre-computed gain value minimizes a sum of per-pixel differences between a normalized color chart and target values of a red-green-blue (RGB) color space adjusted by an RGB value of a black color segment in the normalized color chart;

producing a plurality of color-calibrated images by applying a color transformation matrix to each adjusted image of the plurality of adjusted images;

generating, using the plurality of color-calibrated images, a polygonal mesh simulating geometry of the three-dimensional object;

producing a plurality of partial two-dimensional maps by projecting the plurality of color-calibrated images onto the polygonal mesh;

generating a plurality of masks, wherein each mask of the plurality of masks is associated with a camera of the plurality of cameras, wherein the mask defines a two-dimensional space region that is covered by a field of view of the camera;

blending, using the plurality of masks, the plurality of partial two-dimensional maps; and

generating one or more texture maps representing the three-dimensional object.

2. The method of claim 1 , further comprising:

utilizing the texture maps for creating an interactive video game.

3. The method of claim 1 , further comprising:

computing the color transformation matrix by minimizing a difference between target values in an RGB color space and components of a sample color chart.

4. The method of claim 1 , further comprising:

applying a non-linear tone remapping operation to the plurality of color-calibrated images, wherein a parameter of the non-linear tone remapping operation is computed by minimizing an error function represented by a sum of differences between tone curve values and adjusted color sample values.

5. The method of claim 1 , wherein the texture maps comprise a reflectance map illustrating distribution of specular reflectance on a surface of the three-dimensional object.

6. The method of claim 1 , wherein the texture maps comprise an albedo map produced from a plurality of images acquired under full-on illumination with cross-polarized filters.

7. The method of claim 1 , wherein the texture maps comprise a normal map produced from a plurality of images acquired under partial illumination with parallel-polarized filters.

8. The method of claim 1 , wherein the texture maps comprise a displacement map comprising a plurality of elements, each element defining a displacement values for a corresponding texture element.

9. The method of claim 1 , further comprising:

determining positions of a plurality of markers of a pre-defined color within each input image of the plurality of input images.

10. The method of claim 1 , wherein blending the plurality of partial two-dimensional maps further comprises:

producing a background image by blending a plurality of masked images;

blending a next masked image of the plurality of masked images with the background image to produce a new background image;

iteratively repeating the blending operation for all masked image of the plurality of masked images.

11. A method, comprising:

receiving a plurality of input images of a three-dimensional object, wherein the plurality of input images has been acquired by a plurality of cameras;

producing a plurality of partial two-dimensional maps by projecting the plurality of input images onto a polygonal mesh simulating geometry of the three-dimensional object;

generating a plurality of combined distortion and image maps, wherein each combined distortion and image map comprises a plurality of pixels, wherein intensity of each pixel represents a level of distortion introduced by the respective camera for the pixel;

producing, for each camera, a camera rank map based on the combined distortion and image maps;

generating, using the camera rank maps, a plurality of masks, wherein each mask of the plurality of masks is associated with a camera of the plurality of cameras, wherein the mask defines a two-dimensional space region that is covered by a field of view of the camera;

producing an initial background by blending, using the plurality of masks, the plurality of partial two-dimensional maps;

producing a blended image by iteratively blending each partial two-dimensional map of the plurality of partial two-dimensional maps with the initial background; and

generating, based on the blended image, one or more texture maps representing the three-dimensional object.

12. The method of claim 11 , further comprising:

utilizing the texture maps for creating an interactive video game.

13. The method of claim 11 , wherein the texture maps comprise at least one of: a reflectance map, an albedo map, a normal map, or a displacement map.

14. The method of claim 11 , further comprising:

determining, using the plurality of input images, positions of a plurality of markers of a pre-defined color within each input image of the plurality of input images; and

generating, using the positions of the plurality of markers, the polygonal mesh.

15. The method of claim 11 , further comprising:

applying a non-linear tone remapping operation to the plurality of input images, wherein a parameter of the non-linear tone remapping operation is computed by minimizing an error function represented by a sum of differences between tone curve values and adjusted color sample values.

16. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a processor, cause the processor to:

receive a plurality of input images of a three-dimensional object, wherein the plurality of input images has been acquired by a plurality of cameras using a plurality of illumination and polarization patterns;

produce a plurality of adjusted images by adjusting each input image of the plurality of input images by a pre-computed gain value;

produce a plurality of color-calibrated images by applying a color transformation matrix to each adjusted image of the plurality of adjusted images;

apply a non-linear tone remapping operation to the plurality of color-calibrated images, wherein a parameter of the non-linear tone remapping operation is computed by minimizing an error function represented by a sum of differences between tone curve values and adjusted color sample values;

generate, using the plurality of color-calibrated images, a polygonal mesh simulating geometry of the three-dimensional object;

produce a plurality of partial two-dimensional maps by projecting the plurality of color-calibrated images onto the polygonal mesh;

generate a plurality of masks, wherein each mask of the plurality of masks is associated with a camera of the plurality of cameras, wherein the mask defines a two-dimensional space region that is covered by a field of view of the camera;

blend, using the plurality of masks, the plurality of partial two-dimensional maps; and

generate one or more texture maps representing the three-dimensional object.

17. The computer-readable non-transitory storage medium of claim 16 , further comprising executable instructions that, when executed by the processor, cause the processor to:

utilize the texture maps for creating an interactive video game.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2020
From: LOKEN, KEVIN; MOLTAJI, AMIRHESSAM; HIETANEN, RYAN
To: ELECTRONIC ARTS INC.
Reel/Frame 052055/0274 →
Continuity (2)
Continuation 16036422 · Jul 16, 2018
Related Publication 20200211257A1 · Jul 2, 2020