Methods and systems for motion vectors in 3D graphics
Various methods are provided for generating motion vectors in the context of 3D computer-generated images. An example method includes generating, for each pixel of one or more objects to be rendered in a current frame, a 1-phase motion vector (MV 1 ) and a 0-phase motion vector (MV 0 ), each MV 1 and MV 0 having an associated depth value, to thereby form an MV 1 texture and an MV 0 texture; converting the MV 1 texture to a set of MV 1 blocks and converting the MV 0 texture to a set of MV 0 blocks; and outputting the set of MV 1 blocks and the set of MV 0 blocks for image processing.
1. A method, comprising:
generating, for each pixel of one or more objects to be rendered in a current frame, a 1-phase motion vector (MV 1 ) and a 0-phase motion vector (MV 0 ), each MV 1 and MV 0 having an associated depth value, to thereby form an MV 1 texture and an MV 0 texture;
converting the MV 1 texture to a set of MV 1 blocks and converting the MV 0 texture to a set of MV 0 blocks; and
outputting the set of MV 1 blocks and the set of MV 0 blocks for image processing.
2. The method of claim 1 , wherein each MV 1 represents a change in a position of an object at that pixel from the current frame to a previous frame, and wherein each MV 0 represents a change in a position of the object at that pixel from the previous frame to the current frame.
3. The method of claim 1 , wherein converting the MV 1 texture to the set of MV 1 blocks comprises:
identifying, for each pixel block of a plurality of pixel blocks, a distribution of pixel types in that pixel block; and
converting each pixel block to a respective MV 1 block based on the distribution of pixel types.
4. The method of claim 3 , wherein converting each pixel block to a respective MV 1 block based on the distribution of pixel types comprises:
determining that a first selected pixel block has a non-bimodal distribution such that only ore predominantly mid-range pixels are present, and in response, calculating an average MV 1 over only mid-range pixels in the first selected pixel block;
determining that a second selected pixel block has a bimodal distribution such that the second selected pixel block includes at least one foreground pixel and at least one background pixel, and in response, calculating an average MV 1 over only foreground pixels in the second selected pixel block; and
setting each average MV 1 as an MV 1 value for the respective MV 1 block.
5. The method of claim 1 , wherein converting the MV 0 texture to the set of MV 0 blocks comprises:
identifying, for each pixel block of a plurality of pixel blocks, a distribution of pixel types in that pixel block; and
converting each pixel block to a respective MV 0 block based on the distribution of pixel types.
6. The method of claim 5 , wherein converting each pixel block to a respective MV 0 block based on the distribution of pixel types comprises:
determining that a first selected pixel block has a non-bimodal distribution such that only or predominantly mid-range pixels are present, and in response, calculating an average MV 0 over only mid-range pixels in the first selected pixel block;
determining that a second selected pixel block has a bimodal distribution such that the second selected pixel block includes at least one foreground pixel and at least one background pixel, and in response, calculating an average MV 0 over only foreground pixels in the second selected pixel block; and
setting each average MV 0 as an MV 0 value for the respective MV 0 block.
7. The method of claim 1 , wherein outputting the set of MV 1 blocks and the set of MV 0 blocks comprises outputting the set of MV 1 blocks and the set of MV 0 blocks with frame level metadata and image data for image processing.
8. The method of claim 7 , wherein the image data is separated into objects with motion vectors and objects without motion vectors, and the objects with motion vectors are sent on a separate channel than the objects without motion vectors.
9. The method of claim 8 , wherein objects without motion vectors comprise overlays, particle effects, scene transitions, GUI elements, gun reticules, and/or smoke effects.
10. The method of claim 1 , further comprising applying a double-confirm process to each MV 0 and each MV 1 , wherein the double-confirm process includes confirming each associated depth value.
11. The method of claim 10 , wherein applying the double-confirm process includes adjusting one or more non-confirmed MV 1 s and/or MV 0 s based on a nearest double-confirmed MV 1 and/or MV 0 .
12. The method of claim 10 , wherein applying the double-confirm process includes, for a selected pixel in the current frame, double confirming the MV 1 or MV 0 for the selected pixel responsive to coordinates of the selected pixel in the current frame matching coordinates of that pixel mapped back to a previous frame using the MV 1 or MV 0 for the selected pixel, the coordinates of the selected pixel including a depth coordinate.
13. A system for rendering 3D graphics, comprising:
one or more processors and non-transitory memory allocated to form:
a 3D motion vector module configured to receive a 3D model from a game engine and configured to output a plurality of 1-phase 3D motion vectors (3D MV 1 s ) and a plurality of 0-phase 3D motion vectors (3D MV 0 s ) based on the 3D model, each 3D MV 1 comprising a change in a vertical position, a change in a horizontal position, and a change in a depth position of an object at a respective pixel from a current frame to a previous frame, each MV 0 comprising a change in a vertical position, a change in a horizontal position, and a change in a depth position of an object at a respective pixel from the previous frame to the current frame; and
a block motion vector module configured to generate a 2.5D block MV 1 texture and a 2.5D block MV 0 texture from the plurality of 3D MV 1 s and the plurality of 3D MV 0 s , respectively, and output the 2.5D block MV 1 texture and the 2.5D block MV 0 texture for image processing in order to form an image to be displayed on a display, wherein the 2.5D block MV 1 texture comprises a plurality of MV 1 blocks each formed from a respective subset of the plurality of 3D MV 1 s , and wherein the 2.5D block MV 0 texture comprises a plurality of MV 0 blocks each formed from a respective subset of the plurality of 3D MV 0 s.
14. The system of claim 13 , wherein the 2.5D block MV 1 texture and the 2.5D block MV 0 texture are usable to interpolate a frame between the current frame and the previous frame.
15. The system of claim 13 , wherein the 2.5D block MV 1 texture is formed from a respective subset of double-confirmed and/or corrected 3D MV 1 s and the 2.5D block MV 0 texture is formed from a respective subset of double-confirmed and/or corrected 3D MV 0 s.
16. The system of claim 13 , wherein each MV 1 block comprises an average change in horizontal position for that respective subset of the plurality of 3D MV 1 s , an average change in vertical position for that respective subset of the plurality of 3D MV 1 s , and an average depth value for that respective subset of the plurality of 3D MV 1 s , and wherein each MV 0 block comprises an average change in horizontal position for that respective subset of the plurality of 3D MV 0 s , an average change in vertical position for that respective subset of the plurality of 3D MV 0 s , and an average depth value for that respective subset of the plurality of 3D MV 0 s.
17. A method, comprising:
generating, for each pixel of one or more objects to be rendered in a current frame, a 1-phase motion vector (MV 1 ) and a 0-phase motion vector (MV 0 ), each MV 1 and MV 0 having an associated change in depth, to thereby form an MV 1 texture and an MV 0 texture;
applying a double-confirm process to each MV 0 and each MV 1 , wherein the double-confirm process includes confirming each associated change in depth;
correcting each non-double confirmed MV 0 s and MV 1 s;
converting the MV 1 texture to a set of MV 1 blocks and converting the MV 0 texture to a set of MV 0 blocks, each MV 1 block and each MV 0 block having an associated depth value, including identifying, for each pixel block of a plurality of pixel blocks, a distribution of pixel types in that pixel block and converting each pixel block to a respective MV 0 block and a respective MV 1 based on the distribution of pixel types; and
outputting the set of MV 1 blocks and the set of MV 0 blocks for image processing.
18. The method of claim 17 , wherein each MV 1 represents a change in a position of an object at that pixel from the current frame to a previous frame, and wherein each MV 0 represents a change in a position of the object at that pixel from the previous frame to the current frame.
19. The method of claim 17 , wherein converting each pixel block to a respective MV 0 block and a respective MV 1 block based on the distribution of pixel types comprises:
if the distribution is bimodal such that at least one background pixel and at least one foreground pixel are present, calculating an average MV 0 and an average MV 1 over only foreground pixels in that pixel block, otherwise calculating the average MV 0 and the average MV 1 over only mid-range pixels in that pixel block; and
setting the average MV 0 as an MV 0 value for that MV 0 block and the average MV 1 as an MV 1 value for that MV 1 block.
20. The method of claim 17 , wherein applying the double-confirm process includes, for a selected pixel in the current frame, double confirming the MV 1 or MV 0 for the selected pixel responsive to coordinates of the selected pixel in the current frame matching coordinates of that pixel mapped back to a previous frame using the MV 1 or MV 0 for the selected pixel, the coordinates of the selected pixel including a depth coordinate.