Methods and systems for generating enhanced light texture data

Inventors: Yang Liu (Toronto, CA); Keyi Liu (Markham, CA); Mohamed Ibrahim (Mississauga, CA); Sergei Sachkov (Oakville, CA)

Assignee: HUAWEI TECHNOLOGIES CO., LTD.

G06T15/506 G06T15/04 G06T15/06

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12705824
App. No.: 18/309,283
Granted: Aug 11, 2026
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

System, method and media for processing a first light data structure that specifies, for each of plurality of directions for each of a plurality surface regions corresponding to a scene, respective light measurements, including: applying a trained artificial intelligence (AI) model to the first light texture data structure to generate an enhanced light data structure that specifies, for each of the plurality of directions for each of the plurality surface regions corresponding to the scene, respective enhanced light measurements; and storing an enhanced scene model that includes the enhanced light data structure together with geometric data that maps the enhanced light measurements.

Claims (44)

1 . A computer implemented method for processing a first light data structure that includes light texture data that specifies, for each of plurality of directions for each of a plurality of respective surface regions corresponding to a scene, respective light measurements, comprising:

generating the first light data structure, including defining, for each of the plurality of respective surface regions, a respective local reference frame and a bin structure, the bin structure discretizing the local reference frame into a set of bins, each bin corresponding to a respective range of light directions that intersect the respective surface region, the first light data structure comprises a respective surface region light texture tensor for each of the respective surface regions;

applying a trained artificial intelligence (AI) model to the first light data structure to generate an enhanced light data structure that specifies, for each of the plurality of directions for each of the plurality of respective surface regions corresponding to the scene, respective enhanced light measurements; and

storing an enhanced scene model that includes the enhanced light data structure together with geometric data that maps the enhanced light measurements within the enhanced light data structure to the plurality of respective surface regions.

2 . The method of claim 1 wherein:

the first light data structure includes visibility probability data that specifies, for each of the plurality of respective surface regions corresponding to the scene, respective visibility probability values.

3 . The method of claim 2 wherein:

the first light data structure comprises, for each of the plurality of directions, a respective direction specific first light texture map specifying the respective light measurements for the plurality of respective surface regions for the direction;

the visibility probability data is arranged as a visibility map having the same resolution as the first light texture maps; and

the enhanced light data structure comprises, for each of the plurality of directions, a respective direction specific enhanced light texture map specifying the respective enhanced light measurements for the plurality of respective surface regions for the direction.

4 . The method of claim 3 wherein the direction specific first light texture maps, the visibility map and the direction specific enhanced light texture maps are each formatted as respective two dimensional image files, and the geometric data maps respective pixel locations within the two dimensional image files to the respective surface regions of the plurality of respective surface regions.

5 . The method of claim 4 wherein the AI model comprises at least one of a convolutional auto encoder, a vision transformer, or a recurrent neural network.

6 . The method of claim 1 wherein the AI model comprises at least one or more of: a denoiser, a super-sampler; or an anti-aliasing model.

7 . The method of claim 1 wherein:

the enhanced light texture data structure comprises a respective enhanced surface region light texture tensor for each of the respective surface regions.

8 . The method of claim 1 wherein the respective light measurements each represent a gathered RGB color value light color measurement.

9 . The method of claim 1 wherein the enhanced scene model conforms to a graphics language transmission format (gITF).

10 . The method of claim 1 wherein, for each of the plurality of directions for each of the plurality of respective surface regions, the light measurements represent light data for a respective range of light directions that intersect the respective surface region.

11 . The method of claim 1 comprising wherein generating the first light data structure includes: including:

computing, for each respective surface region, a respective color measurement for each bin of the bin structure, the respective color measurement for each bin being based on a path trace of one or more light ray samples that fall within the respective range of light directions corresponding to the bin, wherein the respective color measurements are used as the respective light measurements;

assembling the first light data structure that indicates the respective local reference frames, bin structures, and respective light measurements for the respective surface regions; and storing the first light data structure.

12 . The method of claim 1 comprising sending the enhanced scene model through a network to a remote rendering device.

13 . The method of claim 12 comprising repeatedly performing the applying, storing and sending in order to support real-time rendering of series of scenes at the rendering device wherein the applying, storying, and sending are performed at a cloud computing platform that is more computationally powerful than the rendering device.

14 . The method of claim 13 comprising:

at the rendering device:

obtaining the enhanced scene model;

rendering a scene image for the scene model based on an input view direction, wherein pixel colors in the rendered scene image are determined based on the light measurements included in the enhanced scene model.

15 . A system comprising one or more processors and one or more non-transitory memories that store executable instructions for the one or more processors, wherein the executable instructions, when executed by the one or more processors, configure the system to perform a method of processing a first light data structure that includes light texture data that specifies, for each of plurality of directions for each of a plurality of respective surface regions corresponding to a scene, respective light measurements, comprising:

applying a trained artificial intelligence (AI) model to the first light data structure to

generate an enhanced light data structure that specifies, for each of the plurality of directions for each of the plurality of respective surface regions corresponding to the scene, respective enhanced light measurements; and

16 . The system of claim 15 wherein:

17 . The system of claim 16 wherein:

the visibility probability data is arranged as a visibility map having the same resolution as the first light texture maps; and

18 . The system of claim 17 wherein the direction specific first light texture maps, the visibility map and the direction specific enhanced light texture maps are each formatted as respective two dimensional image files, and the geometric data maps respective pixel locations within the two dimensional image files to the respective surface regions of the respective plurality of surface regions.

19 . The system of claim 14 wherein the method comprises sending the enhanced scene model through a network to a remote rendering device, and the applying, storing and sending are repeated to support real-time rendering of series of scenes at the rendering device wherein the applying, storying, and sending are performed at a cloud computing platform that is more computationally powerful than the rendering device.

20 . A non-transitory computer readable medium storing computer executable instructions that when executed by one or more processors of a computer system, configure the computer system to perform a method of processing a first light data structure that includes light texture data that specifies, for each of plurality of directions for each of a plurality of respective surface regions corresponding to a scene, respective light measurements, comprising: