IP Library › Granted Patent US 12670660
Granted Patent B2
US 12670660 · App. 18/113,823 · Granted Jun 30, 2026

Method and apparatus with neural scene representation data

Inventors: Donghoon Sagong (Suwon-si, KR); Nahyup Kang (Suwon-si, KR); Jiyeon Kim (Suwon-si, KR); Hyewon Moon (Suwon-si, KR); Seokhwan Jang (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06T15/205G06T7/73G06T15/06
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12670660
App. No.
18/113,823
Granted
Jun 30, 2026
Kind
B2
Abstract

A device including a processor configured to generate, for each of plural query inputs, point information using factors individually extracted from a plurality of pieces of factor data for a corresponding query input and generate pixel information of a pixel position using the point information of points, the plural query inputs being of the points, in a 3D space, on a view direction from a viewpoint toward a pixel position of a two-dimensional (2D) scene.

Claims (53)

1 . An electronic device, comprising:

a processor configured to:

generate, for each of a plurality of query inputs, point information using factors individually extracted from a plurality of pieces of factor data for a corresponding query input; and

generate pixel information of a pixel position using the point information of points,

wherein the plurality of query inputs represent the points, in a three-dimensional (3D) space, along a view direction from a viewpoint toward a pixel position of a two-dimensional (2D) scene, and

wherein the plurality of pieces of factor data is generated based on factorization of a target space tensor derived from a 3D grid representing the 3D space and comprises a first plurality of pieces of factor data related to color and a second plurality of pieces of factor data related to volume density.

2 . The device of claim 1 , wherein the processor is further configured to determine coordinates for the plurality of query inputs, each of the coordinates representing a corresponding point in the 3D space and direction information representing the view direction.

3 . The device of claim 1 , wherein, for the generation of the point information, the processor is configured to generate the point information using a vector outer product of the factors individually extracted from the plurality of pieces of factor data.

4 . The device of claim 1 , wherein, for the generation of the point information, the processor is configured to:

generate, for each query input, color information of a point corresponding to a corresponding query input based on the first plurality of pieces of factor data, and

generate, for each query input, volume density information of a point corresponding to a corresponding query input based on the second plurality of pieces of factor data.

5 . The device of claim 1 , wherein, for the generation of the point information, the processor is configured to:

determine surrounding query inputs corresponding to voxels in a vicinity of a requested voxel among voxels in the 3D space based on an absence of the requested voxel in a piece of the factor data of the plurality of pieces of factor data, and

calculate point information corresponding to the requested voxel using pieces of surrounding point information obtained based on the surrounding query inputs.

6 . The device of claim 1 , wherein the processor is configured to, for the generation of the pixel information, determine a color value of the pixel position to be the pixel information of the pixel position using the point information of the points, and

wherein the processor is further configured to generate an image corresponding to the 2D scene by repeating the determining of the color value for pixel positions included in the 2D scene.

7 . The device of claim 1 , wherein, for the generation of the pixel information, the processor is configured to:

obtain color and volume density curves of points according to the view direction, and

determine the pixel information based on the obtained color and volume density curves.

8 . The device of claim 1 , wherein the processor is configured to generate the pixel information using the plurality of pieces of factor data according to a resolution of a plurality of resolutions.

9 . The device of claim 1 , wherein, for the generation of the point information, the processor is configured to:

generate first point information of first voxels corresponding to a first grid with a first resolution among points along the view direction in the 3D space, using a plurality of pieces of factor data based on the first resolution; and

generate second point information of second voxels corresponding to a second grid with a second resolution, higher than the first resolution, in a vicinity of the first voxels, using a plurality of pieces of factor data based on the second resolution, and

wherein, for the generating of the pixel information, the processor is configured to determine the pixel information based on the first point information and the second point information.

10 . The device of claim 1 , wherein the processor is configured to:

generate a target space tensor representing the 3D space,

factorize the target space tensor into a plurality of tensor sets, and

build the plurality of tensor sets as the plurality of pieces of factor data.

11 . The device of claim 10 , wherein, for the generating of the target space tensor, the processor is configured to compress the 3D space by extracting a portion of voxels in the 3D space.

12 . The device of claim 10 , wherein, for the generating of the target space tensor, the processor is configured to select a voxel having a volume density exceeding a threshold from among voxels in the 3D space.

13 . The device of claim 10 , wherein the processor is configured to build each tensor set by factorizing the target space tensor into tensors of a dimension lower than a dimension of the target space tensor.

14 . The device of claim 10 , wherein each of the plurality of tensor sets is a set of three vectors.

15 . The device of claim 10 , wherein each of the plurality of tensor sets is a set of vectors and matrices.

16 . The device of claim 10 , further comprising a memory,

wherein the processor is configured to:

generate the plurality of pieces of factor data corresponding to each basis tensor of a plurality of basis tensors by merging the plurality of tensor sets based on the plurality of basis tensors; and

store the plurality of pieces of factor data in the memory, and

wherein, for the generating of the point information, the processor is further configured to load the plurality of pieces of factor data.

17 . The device of claim 10 , wherein the generating of the target space tensor is based on an extraction of voxels corresponding to the 3D space from a pre-built neural scene representation (NSR) data cube or an NSR data cube built using a neural network.

18 . The device of claim 1 , wherein the processor is configured to perform the generating of the point information based on the plurality of pieces of factor data and the generating of the pixel information, in parallel with one or more additional respective generations of point information and pixel information for other pixels of an image of the 2D scene.

19 . The device of claim 1 , further comprising a memory storing instructions, which, when executed by the processor, configures the processor to perform the generating of the point information and the generating of the pixel information.

20 . A processor-implemented method, comprising:

generating, for each of a plurality of query inputs, point information using factor values individually extracted from a plurality of pieces of factor data for a corresponding query input; and

generating pixel information of a pixel position using the point information of points,

wherein the plurality of query inputs are of the points, in a three-dimensional (3D) space, along a view direction from a viewpoint toward a pixel position of a two-dimensional (2D) scene image, and

wherein the plurality of pieces of factor data is generated based on factorization of a target space tensor derived from a 3D grid representing the 3D space and comprises a first plurality of pieces of factor data related to color and a second plurality of pieces of factor data related to volume density.

21 . A mobile terminal, comprising:

an input/output interface configured to obtain a plurality of query inputs for a plurality of points, in a three-dimensional (3D) space according to a viewpoint from a view direction for a two-dimensional (2D) scene; and

a processor configured to:

generate, for each of a plurality of query inputs, point information using factors individually extracted from a plurality of pieces of factor data for a corresponding query input; and

generate, for each of the plurality of query inputs, pixel information related to the factor data to generate; and

generate an image corresponding to the 2D scene, and

wherein the plurality of pieces of factor data is generated based on factorization of a target space tensor derived from a 3D grid representing the 3D space and comprises a first plurality of pieces of factor data related to color and a second plurality of pieces of factor data related to volume density.