Motion based adaptive rendering
An apparatus, system and method is provided to determine a motion of pixels in local regions of a scene, classify the motion into a speed category, and make decisions on how to render blocks of pixels. In one implementation the motion in a tile is classified into at least three different speed regimes. If the pixels in a tile are in a quasi-static speed regime, a determination is made whether or not to reuse a fraction of pixels from the previous frame. If the pixels are determined to be in a high speed regime, a decision is made whether or not a sampling rate may be reduced.
1. A method of graphics processing utilizing a graphics processing unit (GPU), the method comprising:
determining, by the GPU, on a tile-by-tile basis, a relative speed of objects as measured in screen space co-ordinates or pixels in a current frame relative to a previous frame, the frames corresponding to images to be displayed on a display device;
classifying, by the GPU, each tile within the current frame into one of at least three speed categories based on the determined relative speed, the at least three categories including a quasi-static speed category, a medium speed category, and a high speed category;
making, by the GPU, a sampling decision for each tile, based at least in part on the speed category associated with each tile, including whether the tile is to be sampled at a full resolution sampling rate of at least one sample per pixel in the current frame or sampled at a lower rate in the current frame; and
rendering, by the GPU, tiles of the current frame.
2. The method of claim 1 , wherein for a tile classified in the quasi-static speed category the method further comprises reusing pixel data from the previous frame by mapping a location of at least one pixel in the current frame to a corresponding pixel in the previous frame and copying pixel data for the corresponding pixel of the previous frame to the current frame.
3. The method of claim 2 , wherein a subset of the pixels from a tile of the previous frame are copied to a tile of the current frame based on a pre-selected ratio of rendered to unrendered pixels.
4. The method of claim 2 , further comprising performing a discrepancy check with respect to pixel color values of the previous frame as a condition for reusing pixel data from the previous frame.
5. The method of claim 1 , wherein the speed is determined from a difference in vertex position data for rendered geometry between the current frame and a previous frame.
6. The method of claim 1 , wherein, for a tile classified in the high speed category, making a sampling decision includes selecting a sampling pattern in which a number of rendered samples is less than the number of pixels associated with the tile.
7. The method of claim 1 , further comprising performing edge detection in depth/Z and wherein the making a sampling decision for each tile is further based on whether edges are detected within individual tiles.
8. The method of claim 7 , wherein:
for a tile in the high speed category not including an edge, selecting a sampling pattern in which a number of samples is less than the number of pixels associated with the tile; and
for a tile in the high speed category including an edge, selecting a sampling pattern in which a number of samples is at least equal to one sample per pixel.
9. The method of claim 1 , wherein making a sampling decision further comprises making a decision for each tile whether or not to perform super-sampling.
10. The method of claim 9 , wherein super-sampling is performed for a tile including an edge.
11. A method of graphics processing utilizing a graphics processing unit (GPU), the method comprising:
determining, by the GPU, on a tile-by-tile basis, a speed of objects as measured in screen space pixels in a current frame relative to a previous frame, the frames to be displayed on a display device;
detecting, by the GPU, on a tile-by-tile basis, whether a tile includes an edge;
classifying, by the GPU based on the determined speed, each tile within the current frame into one of at least three speed categories including a quasi-static speed category, an intermediate speed regime, and at least one high speed regime;
making, by the GPU, a sampling rate decision for individual tiles based on the speed category associated with a tile and whether the tile includes an edge, the sampling decision including determining whether the tile is to be sampled at a full resolution sampling rate of at least one sample per pixel in the current frame or sampled at a lower rate in the current frame; and
rendering, by the GPU, tiles of the current frame.
12. The method of claim 11 , wherein for a tile classified in the quasi-static speed category the method further comprises reusing pixel data from the previous frame by mapping a location of at least one pixel in a current frame to a corresponding pixel in the previous frame and copying pixel data for the corresponding pixel of the previous frame to the current frame.
13. The method of claim 12 , wherein a subset of the pixels from a tile of the previous frame are copied to a tile of the current frame based on a pre-selected ratio of rendered to unrendered pixels.
14. The method of claim 12 , further comprising performing a discrepancy check with respect to pixel color values of the previous frame as a condition for reusing pixel data from the previous frame.
15. The method of claim 11 , wherein for a tile classified in the intermediate speed regime, every pixel is sampled at least once.
16. The method of claim 11 , wherein for a tile classified in the at least one high speed regime selecting a sampling pattern in which a number of samples is less than the number of pixels associated with the tile and performing interpolation to determine a color at pixel locations not sampled.
17. The method of claim 11 , further comprising:
for a tile in the at least one high speed regime not including an edge, selecting a sampling pattern in which a number of samples is less than the number of pixels associated with the tile; and
for a tile in the at least one high speed regime including an edge, selecting a sampling pattern in which a number of samples is at least equal to one sample per pixel.
18. A graphics system including a graphics pipeline, the graphics system comprising:
an adaptive sampling generator; and
a pixel shader coupled to the adaptive sampling generator,
wherein the adaptive sampling generator is configured to determine a required sample rate for each tile from among tiles of an image based at least in part on a speed of pixels of objects in each tile, to select a sample pattern based on the required sample rate, and to determine a sample pattern and a sample rate for an individual tile based on a combination of the speed of the pixels of the objects in the individual tile and whether the individual tile includes an edge, and
wherein a reduced sampling rate is selected for each tile not including an edge and having a speed above a threshold speed.
19. The graphics system of claim 18 , further comprising:
a velocity buffer to compare per-vertex coordinates of vertices of objects in a current frame to corresponding per-vertex coordinates in a previous frame, and to determine a per-pixel velocity of pixels in each tile;
an edge detector to detect a presence of edges in each tile; and
a sample generator to determine, for each tile to be shaded, a sample pattern and sampling rate based on a speed associated with pixels in each tile and the presence of edges.
20. The graphics system of claim 18 , further comprising an advection unit and a reconstruction unit, wherein, for a tile having a velocity below a quasi-static speed limit, a sample pattern having a reduced sampling rate is selected and the reconstruction unit fills in missing pixel data by reusing pixel data from a previous frame via advection.
21. The graphics system of claim 20 , wherein the reconstruction unit identifies a location of a pixel in a previous frame by mapping a change in pixel coordinates.
22. The graphics system of claim 20 , wherein for the tile having a velocity below the quasi-static speed limit a preselected subset of the pixels of the tile are rendered and the remainder of the pixels are reused from the previous frame based on a pre-selected ratio of rendered to unrendered pixels.
23. A method of graphics processing utilizing a graphics processing unit (GPU), the method comprising:
adaptively rendering, by the GPU, at least one tile of a current frame based on speed determined by:
comparing the current frame to a previous frame;
determining, by the GPU, for the at least one tile of the current frame a relative speed of objects as measured in screen space co-ordinates or pixels in the current frame relative to a previous frame;
classifying, by the GPU based on the determined relative speed, each tile within the current frame into one of at least three speed categories including a quasi-static speed category, a medium speed category, and a high speed category;
making, by the GPU, a sampling decision for each tile, based at least in part on the speed category associated with each tile, including whether the tile is to be sampled at a full resolution sampling rate of at least one sample per pixel in the current frame or sampled at a lower rate in the current frame; and
rendering, by the GPU, tiles of the current frame,
wherein the frames correspond to images to be displayed on a display device.