Systems and methods for player input motion compensation by anticipating motion vectors and/or caching repetitive motion vectors
Systems and methods for reducing latency through motion estimation and compensation techniques are disclosed. The systems and methods include a client device that uses transmitted lookup tables from a remote server to match user input to motion vectors, and tag and sum those motion vectors. When a remote server transmits encoded video frames to the client, the client decodes those video frames and applies the summed motion vectors to the decoded frames to estimate motion in those frames. In certain embodiments, the systems and methods generate motion vectors at a server based on predetermined criteria and transmit the generated motion vectors and one or more invalidators to a client, which caches those motion vectors and invalidators. The server instructs the client to receive input from a user, and use that input to match to cached motion vectors or invalidators. Based on that comparison, the client then applies the matched motion vectors or invalidators to effect motion compensation in a graphic interface. In other embodiments, the systems and methods cache repetitive motion vectors at a server, which transmits a previously generated motion vector library to a client. The client stores the motion vector library, and monitors for user input data. The server instructs the client to calculate a motion estimate from the input data and instructs the client to update the stored motion vector library based on the input data, so that the client applies the stored motion vector library to initiate motion in a graphic interface prior to receiving actual motion vector data from the server. In this manner, latency in video data streams is reduced.
1. A computer-implemented method for processing motion vectors, the method comprising:
transmitting a previously generated motion vector from a server to a client;
receiving, at the client, the previously generated motion vector;
at the client, monitoring for input data;
at the client, calculating a motion estimate based at least in part on the input data;
at the client, applying the previously generated motion vector as part of client-side player input motion compensation, wherein the client initiates motion in a graphic interface;
at the client, after starting the applying the previously generated motion vector, receiving an actual motion vector, from the server, that indicates actual motion in encoded video in reaction to the input data; and
at the client, compensating for the applying the previously generated motion vector, as part of the client-side player input motion compensation, in place of the actual motion vector.
2. The computer-implemented method of claim 1 , further comprising, at the client, storing the previously generated motion vector in a cache.
3. The computer-implemented method of claim 1 , further comprising, at the client, scaling the previously generated motion vector depending on playback rate.
4. The computer-implemented method of claim 1 , further comprising receiving, at the client, a context update to enable or disable application of the previously generated motion vector.
5. A computer-implemented method for processing motion vectors, the method comprising:
receiving, at a client, a previously generated motion vector from a server;
at the client, monitoring for input data;
at the client, calculating a motion estimate based at least in part on the input data;
at the client, applying the previously generated motion vector as part of client-side player input motion compensation, wherein the client initiates motion in a graphic interface;
at the client, after starting the applying the previously generated motion vector, receiving an actual motion vector, from the server, that indicates actual motion in encoded video in reaction to the input data; and
at the client, compensating for the applying the previously generated motion vector, as part of the client-side player input motion compensation, in place of the actual motion vector.
6. The computer-implemented method of claim 5 , further comprising receiving, along with the encoded video, a correlation tag, wherein the correlation tag is associated with the input data.
7. The computer-implemented method of claim 6 , wherein the correlation tag associated with the input data signals the client to stop the applying the previously generated motion vector or undo a previous player input motion compensation operation.
8. The computer-implemented method of claim 7 , wherein the previous player input motion compensation operation is undone using a blending technique.
9. The computer-implemented method of claim 5 , further comprising, at the client, scaling the previously generated motion vector depending on playback rate.
10. The computer-implemented method of claim 5 , wherein the previously generated motion vector is received before the input data is received, wherein the monitoring for input data is performed during play of a video game, and wherein the actual motion vector is generated and received during the play of the video game.
11. The computer-implemented method of claim 5 , wherein the compensating includes:
calculating one or more correction motion vectors based at least in part on difference between the previously generated motion vector and the actual motion vector; and
performing motion compensation using the one or more correction motion vectors.
12. A system for processing motion vectors, the system comprising one or more processing units and memory, wherein the system implements a client configured to perform operations comprising:
receiving, at the client, a previously generated motion vector from a server;
at the client, monitoring for input data;
at the client, calculating a motion estimate based at least in part on the input data;
at the client, applying the previously generated motion vector as part of client-side player input motion compensation, wherein the client initiates motion in a graphic interface;
at the client, after starting the applying the previously generated motion vector, receiving an actual motion vector, from the server, that indicates actual motion in encoded video in reaction to the input data; and
at the client, compensating for the applying the previously generated motion vector, as part of the client-side player input motion compensation, in place of the actual motion vector.
13. The system of claim 12 , wherein the operations further comprise receiving, along with the encoded video, a correlation tag, the correlation tag being associated with the input data.
14. The system of claim 13 , wherein the correlation tag associated with the input data signals the client to stop the applying the previously generated motion vector or undo a previous player input motion compensation operation.
15. The system of claim 14 , wherein the previous player input motion compensation operation is undone using a blending technique.
16. The system of claim 12 , wherein the operations further comprise, at the client, scaling the previously generated motion vector depending on playback rate.
17. The system of claim 12 , wherein the previously generated motion vector is received before the input data is received, wherein the monitoring for input data is performed during play of a video game, and wherein the actual motion vector is generated and received during the play of the video game.
18. The system of claim 12 , wherein the compensating includes:
calculating one or more correction motion vectors based at least in part on difference between the previously generated motion vector and the actual motion vector; and
performing motion compensation using the one or more correction motion vectors.
19. A system for processing motion vectors, the system comprising one or more processing units and memory, wherein the system implements a server configured to perform operations comprising:
from the server, transmitting a previously generated motion vector to a client, the previously generated motion vector indicating motion, in a graphic interface of the client, for client-side player input motion compensation in reaction to input data;
receiving, at the server, the input data from the client;
at the server, encoding video, the encoded video including an actual motion vector that indicates actual motion in encoded video in reaction to the input data; and
from the server, transmitting the encoded video to the client for decoding that includes compensating for applying the previously generated motion vector, as part of the client-side player input motion compensation, in place of the actual motion vector.
20. The system of claim 19 , wherein the operations further comprise transmitting, along with the encoded video, a correlation tag, the correlation tag being associated with the input data.