Image processing for display of multi-dimensional objects
In a storage device are stored, as reference maps, a background map, a color map of an object, and a height map of the object as hierarchical data having a plurality of resolutions. An image processing device loads, when necessary, portions thereof which are required for rendering into a main memory according to movement of a viewpoint, etc., decodes and decompresses portions thereof, and develops the resulting pieces of data in a memory inside a GPU. The GPU refers to these pieces of data to render a display image through ray tracing.
1 . A device comprising:
one or more processors; and
one or more computer-readable media having stored thereon instructions that, when executed with the one or more processors, cause the device to, at least:
load first portions of hierarchical data representing, in a first resolution, both (a) a first color map representing a distribution of color values over a surface of a first object to be displayed and (b) a first height map representing a distribution of heights from a first base shape surface to the surface of the first object, the first portions being required for rendering of the first object;
load second portions of the hierarchical data representing, in a second resolution different from the first resolution, both (a) a second color map representing a distribution of color values over a surface of a second object to be displayed and (b) a second height map representing a distribution of heights from a surface of a second base shape to the surface of the second object, the second object incorporating the second base shape arranged on the surface of the first object, the second portions being required for rendering of the second object;
obtain a first point on the first object at which a first ray in ray tracing arrives, by referring to the first height map and based on the first resolution of the first height map, and acquire a first color value of the first object at the first point by referring to the first color map and based on the first resolution of the first color map, to determine a first pixel value of a display image;
obtain a second point on the second object at which a second ray in the ray tracing arrives, by referring to the second height map and based on the second resolution of the second height map, and acquire a second color value of the second object at the second point by referring to the second color map and based on the second resolution of the second color map, to determine a second pixel value of the display image; and
output data of the display image including a rendering of at least a portion of the first object and at least a portion of the second object.
2 . The device according to claim 1 , wherein the instructions further cause the device to:
determine a region in a three-dimensional space formed by a plane of one of the first color map, the first height map, or an axis of resolution, on a basis of a viewpoint and a line of sight for the first object, wherein
data of each of the first color map and the first height map is loaded corresponding to the region.
3 . The device according to claim 2 , wherein
the hierarchical data has a link region at which a switch to other data is set in the three-dimensional space, and wherein the instructions further cause the device to
load the other data when the link region is included in the region.
4 . The device according to claim 3 , wherein the instructions further cause the device to:
acquire, as the other data, a third color map and a third height map generated separately with respect to a portion of the first object, and
determine a third pixel value representing the portion of the first object by using the other data.
5 . The device according to claim 3 , wherein the instructions further cause the device to:
acquire, as the other data, data for representing a portion of the first object with a model of a different type, and
determine a third pixel value representing the portion of the first object by using the other data.
6 . The device according to claim 1 , wherein, in at least any one of the first color map and the first height map, changes in a parameter other than resolution are given to data represented by layers of the hierarchical data.
7 . The device according to claim 1 , wherein
the first color map includes a distribution of a reflection coefficient over the surface of the first object, and wherein the instructions further cause the device to:
correct the first color value on a basis of the reflection coefficient.
8 . The device according to claim 1 , wherein the instructions further cause the device to:
determine the first pixel value on a basis of color values and heights obtained by filtering of data of a range on the first color map or the first height map, the range corresponding to each of pixel regions on a view screen matching a current viewpoint and line of sight.
9 . The device according to claim 1 , wherein the instructions further cause the device to:
decode and decompress a portion of loaded compression-encoded data, and store resulting data in an internal memory of a graphics processing unit.
10 . The device according to claim 1 , wherein the distribution of heights of the first height map is different from the distribution of heights of the second height map.
11 . The device according to claim 1 , wherein the first base shape is different from the second base shape.
12 . The device according to claim 1 , wherein the second ray in the ray tracing does not pass through the first object and the second object is a background of the display image.
13 . The device according to claim 1 , wherein at least one of the first resolution and the second resolution changes based on a change in a viewpoint associated with the display image.
14 . The device according to claim 1 , wherein the first object and the second object correspond to a same object viewed from different viewpoints.
15 . The device according to claim 1 , wherein a shape of at least one of the first object and the second object changes based on a change in a viewpoint associated with the display image.
16 . The device according to claim 1 , wherein a color of at least one of the first object and the second object changes based on a change in a viewpoint associated with the display image.
17 . The device according to claim 1 , wherein at least one of the first height map and the second height map changes based on a change in a viewpoint associated with the display image.
18 . The device according to claim 1 , wherein at least one of the first color map and the second color map changes based on a change in a viewpoint associated with the display image.
19 . A method comprising:
loading, into a memory, first portions of hierarchical data representing, in a first resolution, both (a) a first color map representing a distribution of color values over a surface of a first object to be displayed and (b) a first height map representing a distribution of heights from a first base shape surface to the surface of the first object, the first portions being required for rendering of the first object;
loading second portions of the hierarchical data representing, in a second resolution different from the first resolution, both (a) a second color map representing a distribution of color values over a surface of a second object to be displayed and (b) a second height map representing a distribution of heights from a surface of a second base shape to the surface of the second object, the second object incorporating the second base shape arranged on the surface of the first object, the second portions being required for rendering of the second object;
obtaining a first point on the first object at which a first ray in ray tracing arrives, by referring to the first height map and based on the first resolution of the first height map, and acquiring a first color value of the first object at the first point by referring to the first color map and based on the first resolution of the first color map, to determine a first pixel value of a display image;
obtaining a second point on the second object at which a second ray in the ray tracing arrives, by referring to the second height map and based on the second resolution of the second height map, and acquiring a second color value of the second object at the second point by referring to the second color map and based on the second resolution of the second color map, to determine a second pixel value of the display image; and
outputting data of the display image including a rendering of at least a portion of the first object and at least a portion of the second object.
20 . A non-transitory, computer-readable storage medium containing a computer program, which when executed by a computer, causes the computer to carry out actions, comprising:
loading first portions of hierarchical data representing, in a first resolution, both (a) a first color map representing a distribution of color values over a surface of a first object to be displayed and (b) a first height map representing a distribution of heights from a first base shape to the surface of the first object, the first portions being required for rendering of the first object;
loading second portions of the hierarchical data representing, in a second resolution different from the first resolution, both (a) a second color map representing a distribution of color values over a surface of a second object to be displayed and (b) a second height map representing a distribution of heights from a surface of a second base shape to the surface of the second object, the second object incorporating the second base shape arranged on the surface of the first object, the second portions being required for rendering of the second object;
obtaining a first point on the first object at which a first ray in ray tracing arrives, by referring to the first height map and based on the first resolution of the first height map, and acquiring a first color value of the first object at the first point by referring to the first color map and based on the first resolution of the first color map, to determine a first pixel value of a display image;
obtaining a second point on the second object at which a second ray in the ray tracing arrives, by referring to the second height map and based on the second resolution of the second height map, and acquiring a second color value of the second object at the second point by referring to the second color map and based on the second resolution of the second color map, to determine a second pixel value of the display image; and
outputting data of the display image including a rendering of at least a portion of the first object and at least a portion of the second object.