Methods and systems for dual channel transfer of game
Various methods are provided for generating motion vectors in the context of 3D computer-generated images. An example method includes generating, based on a current frame to be rendered, a block-level motion vector with depth (MVD) texture comprising a set of block-level motion vectors each having an associated depth value, outputting the block-level MVD texture for image processing on a first channel, and outputting at least a portion of the current frame on a second channel.
1. A method, comprising:
generating, based on a current frame to be rendered, a block-level motion vector with depth (MVD) texture that comprises a set of block-level motion vectors each having an associated depth value; and
outputting the block-level MVD texture for image processing on a first channel and outputting at least a portion of the current frame on a second channel, wherein the set of block-level motion vectors includes a first set of block-level 0-phase motion vectors with depth (MVD0s) and a second set of block-level 1-phase motion vectors with depth (MVD1s), each block-level MVD0 and MVD1 having an associated depth value, wherein each block-level MVD1 is generated from a respective group of pixel-level MVD1s, each pixel-level MVD1 representing a change in a position of an object at that pixel from the current frame to a previous frame, and wherein each block-level MVD0 is generated from a respective group of pixel-level MVD0s, each pixel-level MVD0 representing a change in a position of the object at that pixel from the previous frame to the current frame.
2. The method of claim 1 , wherein generating each block-level MVD1 comprises:
identifying, for each pixel block in a plurality of pixel blocks, a distribution of pixel types in that pixel block; and
converting each pixel block to a respective MVD1 block based on the distribution of pixel types.
3. The method of claim 2 , wherein converting each pixel block to a respective MVD1 block based on the distribution of pixel types comprises:
determining that a first selected pixel block has a non-bimodal distribution such that only or predominantly mid-range pixels are present, and in response, calculating an average MVD1 over only mid-range pixels in the first selected pixel block;
determining that a second selected pixel block has a bimodal distribution such that the second selected pixel block includes at least one foreground pixel and at least one background pixel, and in response, calculating an average MVD1 over only foreground pixels in the second selected pixel block; and
setting each average MVD1 as an MVD1 value for the respective MVD1 block.
4. The method of claim 1 , wherein generating each block-level MVD0 comprises:
identifying, for each pixel block in a plurality of pixel blocks, a distribution of pixel types in that pixel block; and
converting each pixel block to a respective MVD0 block based on the distribution of pixel types.
5. The method of claim 4 , wherein converting each pixel block to a respective MVD0 block based on the distribution of pixel types comprises:
determining that a first selected pixel block has a non-bimodal distribution such that only or predominantly mid-range pixels are present, and in response, calculating an average MVD0 over only mid-range pixels in the first selected pixel block;
determining that a second selected pixel block has a bimodal distribution such that the second selected pixel block includes at least one foreground pixel and at least one background pixel, and in response, calculating an average MVD0 over only foreground pixels in the second selected pixel block; and
setting each average MVD0 as an MVD0 value for the respective MVD0 block.
6. The method of claim 1 , wherein the current frame includes objects with motion vectors and objects without motion vectors, and wherein generating the block-level MVD texture comprises generating the block-level MVD texture based only information relating to the objects with motion vectors.
7. The method of claim 6 , wherein outputting the block-level MVD texture for image processing on the first channel and outputting at least the portion of the current frame on the second channel comprises outputting the block-level MVD texture, the objects with motion vectors, and the objects without motion vectors across two or more physically and/or logically separated channels.
8. The method of claim 7 , wherein objects without motion vectors comprise particle effects and static user interface elements, wherein outputting the block-level MVD texture, the objects with motion vectors, and the objects without motion vectors across two or more physically and/or logically separated channels comprises outputting the block-level MVD texture and the objects with motion vectors on the first channel and outputting the objects without motion vectors on the second channel, and wherein the block-level MVD texture is not output on the second channel.
9. The method of claim 1 , wherein the current frame includes objects with motion vectors and objects without motion vectors, and wherein generating the block-level MVD texture comprises generating a masked block-level MVD texture, the masked block-level MVD texture generated by masking MVDs associated with objects without motion vectors.
10. The method of claim 9 , wherein outputting the block-level MVD texture for image processing on the first channel and outputting at least the portion of the current frame on the second channel comprises outputting the block-level MVD and a duplicate of the current frame on the first channel and outputting the current frame on the second channel.
11. A method for sending motion vectors (MVs) to be used in frame rate conversion, comprising:
receiving, from a game engine configured to output a current frame of a video game to be rendered on a display device, a pixel-level 1-phase motion vector (MV1) texture comprising, for each pixel, a change in a position of an object at that pixel from the current frame to a previous frame, a pixel-level 0-phase motion vector (MV0) texture comprising, for each pixel, a change in a position of the object at that pixel from the previous frame to the current frame, and a depth texture of one or more objects of the current frame;
generating, based on the pixel-level MV1 texture, the pixel-level MV0 texture, and the depth texture, a block-level motion vector with depth (MVD) texture comprising a set of block-level motion vectors each having an associated depth value; and
outputting the block-level MVD texture, the one or more objects of the current frame, and additional objects not having motion vectors of the current frame for image processing across two or more physically and/or logically separated channels, including outputting the block-level MVD texture along with the one or more objects of the current frame on a first channel and outputting the additional objects on a second channel.
12. The method of claim 11 , wherein the additional objects not having motion vectors include particle effects and static user interface elements.
13. The method of claim 11 , wherein an operating system graphical user interface is blended with the additional objects on the second channel.
14. The method of claim 11 , wherein the block-level MVD texture and the one or more objects of the current frame are usable to interpolate a frame between the current frame and the previous frame generated by the game engine.
15. The method of claim 14 , wherein the additional objects not having motion vectors are blended with the interpolated frame.
16. A system for motion vectors (MVs) to be used in frame rate conversion, comprising:
one or more processors; and
memory storing instructions executable by the one or more processors to:
receive, from a game engine configured to output a current frame of a video game to be rendered on a display device, a pixel-level 1-phase motion vector (MV1) texture comprising, for each pixel, a change in a position of an object at that pixel from the current frame to a previous frame, a pixel-level 0-phase motion vector (MV0) texture comprising, for each pixel, a change in a position of the object at that pixel from the previous frame to the current frame, and a depth texture of one or more objects of the current frame;
generate, a block-level motion vector with depth (MVD) texture comprising a set of block-level motion vectors each having an associated depth value based on the pixel-level MV1 texture, the pixel-level MV0 texture, and the depth texture; and
output the block-level MVD texture, the one or more objects of the current frame, and additional objects not having motion vectors of the current frame for image processing across two or more physically and/or logically separated channels, including outputting the one or more objects of the current frame and the additional objects on a first physical channel and outputting the block-level MVD texture on a second physical channel.
17. The system of claim 16 , wherein the block-level MVD texture, the one or more objects of the current frame, and the additional objects not having motion vectors of the current frame are output to an image processing module configured to interpolate a frame between the current frame and a previous frame generated by the game engine using the block-level MVD texture.