Split compute reprojection
In various examples there is a computer-implemented method performed by a rendering device. A depth image is rendered from a 3D model according to a predicted pose. The depth image is divided into a plurality of tiles. An error map is computed comprising, for each tile of the plurality of tiles, a planar deviation error value that represents a geometric complexity of the tile and comprises an extent to which a geometry of the tile deviates from a plane. The error map, or information derived from the error map, is sent to a display device.
1 . A method for planar deviation image reprojection, performed by a rendering device that is remote from a display device and has greater processing resources than the display device, the method comprising:
rendering, by the rendering device, a color image and a depth image from a three-dimensional (3D) model according to a predicted pose of the display device;
transmitting, via a network, the color image and the depth image from the rendering device to the display device;
performing, at the rendering device, computation of geometric data for use in reprojection of the color image at the display device, the computation comprising:
dividing the depth image into a plurality of tiles; and
computing geometric data comprising an error map, a budget map derived from the error map, or a mesh based on the budget map;
transmitting, via the network, the geometric data to the display device,
wherein:
when the geometric data comprises the error map, the error map comprises, for a tile of the plurality of tiles, a planar deviation error value based on an extent to which a geometry of the tile deviates from a plane;
when the geometric data comprises the budget map, the budget map comprises, for the tile, a count of primitives budgeted for the tile based on the planar deviation error value for the tile; and
when the geometric data comprises the mesh, the mesh is formed of polygons tessellated according to the budget map and based on the depth image.
2 . The method as claimed in claim 1 , wherein the geometric data comprises the budget map.
3 . The method as claimed in claim 1 , wherein the predicted pose is computed from pose data received from the display device or is received from the display device.
4 . The method as claimed in claim 1 , wherein the geometric data further comprises a cross-visibility map specifying that a tile of the plurality of tiles depicts a disoccluded part of a scene which is not visible in a second image rendered from the 3D model.
5 . The method as claimed in claim 4 , wherein the second image is rendered according to a previous predicted pose or according to a stereo viewpoint.
6 . The method as claimed in claim 4 , further comprising compressing the cross-visibility map and sending the compressed cross-visibility map to the display device.
7 . The method of claim 1 , wherein the geometric data comprises the error map.
8 . The method of claim 1 , wherein the geometric data comprises the mesh.
9 . The method of claim 1 , wherein the geometric data is compressed prior to the transmitting of the geometric data to the display device.
10 . A rendering device for assisting in planar deviation image reprojection, the rendering device being remote from a display device, the rendering device comprising a processor and a memory storing instructions executable by the processor to:
render, by the rendering device, a color image and a depth image from a three-dimensional (3D) model according to a predicted pose of the display device;
transmit, via a network, the color image and the depth image from the rendering device to the display device;
perform, at the rendering device, computation of geometric data for use in the reprojection of the color image at the display device, the computation comprising:
divide the depth image into a plurality of tiles; and
compute geometric data comprising an error map, a budget map derived from the error map, or a mesh based on the budget map; and
transmit, via the network, the geometric data to the display device.
11 . The rendering device of claim 10 , wherein:
when the geometric data comprises the error map, the error map comprises, for a tile of the plurality of tiles, a planar deviation error value based on an extent to which a geometry of the tile deviates from a plane;
when the geometric data comprises the budget map, the budget map comprises, for the tile, a count of primitives budgeted for the tile based on the planar deviation error value for the tile; and
when the geometric data comprises the mesh, the mesh is formed of polygons tessellated according to the budget map and based on the depth image.
12 . The rendering device of claim 10 , wherein the predicted pose is computed from pose data received from the display device or is received from the display device.
13 . The rendering device of claim 10 , wherein the plane is a plane fitted to values in a depth buffer storing the depth image.
14 . The rendering device of claim 13 , wherein the depth buffer stores perspectively projected depth values of the depth image.
15 . The rendering device of claim 10 , wherein the geometric data comprises the error map.
16 . The rendering device of claim 10 , wherein the geometric data comprises the mesh.
17 . The rendering device of claim 10 , wherein the geometric data is compressed prior to the transmitting of the geometric data to the display device.
18 . A computer-readable storage medium storing instructions executable by a processing apparatus to perform operations comprising:
rendering a color image and a depth image from a three-dimensional (3D) model according to a predicted pose;
transmitting the color image and the depth image to a display device;
generating geometric data, the generating of the geometric data comprising:
dividing the depth image into a plurality of tiles; and
computing a cross-visibility map specifying that a tile of the plurality of tiles includes a disoccluded part of a scene which is not visible in a second image rendered from a different perspective of the 3D model; and
sending the geometric data to the display device, the geometric data comprising the cross-visibility map.
19 . The computer-readable storage medium of claim 18 , wherein the different perspective is based on a previous predicted pose or a stereo viewpoint.
20 . The computer-readable storage medium of claim 18 , wherein the transmitting further comprising compressing the cross-visibility map and sending the compressed cross-visibility map to the display device.