IP Library Granted Patent US 11,915,397
Granted Patent B2
US 11,915,397 · App. 17/956,925 · Granted Feb 27, 2024

Rendering an image of a 3-D scene

Inventors: Szabolcs Csefalvay (Hertfordshire, GB); James Imber (Hertfordshire, GB); David Walton (Hertfordshire, GB); Insu Yu (Hertfordshire, GB)
Assignee: Imagination Technologies Limited
G06T5/002G06T3/40G06T5/20G06T5/50G06T15/06G06T15/506G06T2207/20016G06T2207/20212G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,915,397
App. No.
17/956,925
Granted
Feb 27, 2024
Kind
B2
Abstract

A method of rendering an image of a 3-D scene includes rendering a noisy image; and obtaining one or more guide channels. For each of a plurality of local neighbourhoods, the method comprises: calculating the parameters of a model that approximates the noisy image as a function of the one or more guide channels, and applying the calculated parameters to produce a denoised image. Tiling is used when calculating the parameters of the model.

Claims (68)

1. A method of rendering an image of a 3-D scene, the method comprising:

rendering a noisy image;

obtaining one or more guide channels; and

for each of a plurality of local neighbourhoods:

calculating the parameters of a model that approximates the noisy image as a function of the one or more guide channels, and

applying the calculated parameters to produce a denoised image,

wherein calculating the parameters of the model comprises:

defining a first tile, defining respective first contiguous portions of the noisy image and the one or more guide channels, each comprising a first plurality of pixels;

defining a second tile, defining respective second contiguous portions of the noisy image and the one or more guide channels, each comprising a second plurality of pixels;

calculating a first outer product between each pixel in the one or more guide channels and itself; and

calculating a second outer product between each pixel in the one or more guide channels and the corresponding pixel in the noisy image,

wherein the first outer product and second outer product are calculated for pixels in the first tile either (i) before the second tile or (ii) concurrently with the second tile.

2. The method of claim 1 , wherein rendering the noisy image comprises rendering by path tracing.

3. The method of claim 1 , wherein the noisy image comprises (i) indirect lighting in the scene or (ii) an ambient occlusion image.

4. The method of claim 3 , wherein the denoised image is a denoised indirect lighting image or a denoised ambient occlusion image, and wherein the method further comprises:

obtaining a direct lighting image; and

combining the denoised image with the direct lighting image to produce a global illumination image.

5. The method of claim 4 , wherein obtaining the direct lighting image comprises rendering it by ray-tracing or rendering it by rasterization.

6. The method of claim 3 , wherein the noisy image is a noisy global illumination image, comprising direct and indirect lighting in the scene, whereby the denoised image is a denoised global illumination image.

7. The method of claim 4 , further comprising combining the global illumination image or the denoised global illumination image with a surface reflectance image to produce a rendered image of the 3-D scene.

8. The method of claim 1 , wherein obtaining the one or more guide channels comprises rendering by rasterization.

9. The method of claim 1 , wherein the guide channels comprise any one or any combination of two or more of:

depth information of objects in the 3-D scene;

information identifying materials of objects in the 3-D scene;

surface reflectances of objects in the 3-D scene;

shadows in the 3-D scene; and

surface normals of objects in the 3-D scene.

10. The method of claim 1 , wherein calculating the parameters of the model comprises:

calculating a first outer product between each pixel in the one or more guide channels and itself;

calculating a second outer product between each pixel in the one or more guide channels and the corresponding pixel in the noisy image;

blurring the first outer products to calculate a first moment matrix for each local neighbourhood;

blurring the second outer products to calculate a second moment matrix for each local neighbourhood; and

calculating the parameters of the model for each local neighbourhood, comprising calculating an inverse matrix of the first moment matrix, and calculating a product of the inverse matrix and the second moment matrix.

11. The method of claim 10 , wherein blurring the first outer products comprises calculating a first multiscale pyramid from the first outer products and calculating the first moment matrix based on the first multiscale pyramid; and/or

wherein blurring the second outer products comprises calculating a second multiscale pyramid from the second outer products and calculating the second moment matrix based on the second multiscale pyramid.

12. The method of claim 10 , wherein the blurring comprises separable filtering in horizontal and vertical directions.

13. The method of claim 10 , wherein the blurring comprises filtering using an anisotropic 2-D filter.

14. The method of claim 11 , wherein the one or more guide channels include surface normals of objects in the 3-D scene, and wherein the blurring comprises:

for each local neighbourhood,

determining a major axis and minor axis of a 2-D filter, based on the surface normal of the object at the centre of the neighbourhood;

selecting a level of the multiscale pyramid, based on the length of the minor axis; and

sampling the selected level of the multiscale pyramid along the major axis.

15. The method of claim 10 , wherein the blurring comprises one of:

IIR filtering; and

filtering with a running box filter.

16. The method of claim 10 , comprising:

defining a first outer product tile, defining a first contiguous portion of the first outer product and a respective first contiguous portion of the second outer product, each comprising a first plurality of pixels; and

defining a second outer product tile, defining a second contiguous portion of the first outer product and a respective second contiguous portions of the second outer product, each comprising a second plurality of pixels,

wherein the first moment matrix and second moment matrix are calculated for the first tile either (i) before the second tile or (ii) concurrently with the second tile.

17. The method of claim 10 , further comprising normalizing and/or regularizing one or both of the first moment matrix and the second moment matrix.

18. A non-transitory computer readable storage medium having stored thereon computer readable code configured to cause the method as set forth in claim 1 to be performed when the code is run.

19. A method of rendering an image of a 3-D scene, the method comprising:

rendering a noisy image;

obtaining one or more guide channels; and

for each of a plurality of local neighbourhoods:

calculating the parameters of a model that approximates the noisy image as a function of the one or more guide channels, and

applying the calculated parameters to produce a denoised image,

wherein calculating the parameters of the model comprises:

calculating a first outer product between each pixel in the one or more guide channels and itself;

calculating a second outer product between each pixel in the one or more guide channels and the corresponding pixel in the noisy image;

blurring the first outer products to calculate a first moment matrix for each local neighbourhood;

blurring the second outer products to calculate a second moment matrix for each local neighbourhood; and

calculating the parameters of the model for each local neighbourhood, comprising calculating an inverse matrix of the first moment matrix, and calculating a product of the inverse matrix and the second moment matrix,

the method comprising, when blurring the first and second outer products:

defining a first outer product tile, defining a first contiguous portion of the first outer product and a respective first contiguous portion of the second outer product, each comprising a first plurality of pixels; and

defining a second outer product tile, defining a second contiguous portion of the first outer product and a respective second contiguous portions of the second outer product, each comprising a second plurality of pixels,

wherein the first moment matrix and second moment matrix are calculated for the first tile either (i) before the second tile or (ii) concurrently with the second tile.

20. A non-transitory computer readable storage medium having stored thereon computer readable code configured to cause the method as set forth in claim 19 to be performed when the code is run.

Assignments (2)
SECURITY INTEREST Recorded Jul 31, 2024
From: IMAGINATION TECHNOLOGIES LIMITED
To: FORTRESS INVESTMENT GROUP (UK) LTD
Reel/Frame 068221/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2024
From: CSEFALVAY, SZABOLCS; IMBER, JAMES; WALTON, DAVID; YU, INSU
To: IMAGINATION TECHNOLOGIES LIMITED
Reel/Frame 066243/0201 →