Cloud-based input latency measurement
Techniques for measuring cloud-based input latency are described herein. In accordance with various embodiments, a server including one or more processors and a non-transitory memory receives from a client an input event corresponding to a request for a TV user interface (UI). The server adds a tag to each of a set of screens corresponding to the TV UI and records event data corresponding to the input event prior to rendering the set of screens into a set of rendered frames. The server additionally decodes the tag in the set of rendered frames to generate metadata that associate the event data with the set of rendered frames prior to encoding the set of rendered frames into a set of encoded frames. The server also transmits to the client the set of encoded frames and the metadata.
1 . A method comprising:
at a server including one or more processors and a non-transitory memory:
receiving an input event from a client requesting a TV user interface (UI);
tagging frames corresponding to the TV UI according to a tagging scheme prior to rendering the frames into rendered tagged frames, wherein the tagging scheme specifies a rotation pattern for tagging the frames responsive to receiving input events, and the tagging scheme uses at least one pixel following the rotation pattern;
generating metadata according to the rotation pattern to associate the input event with the rendered tagged frames prior to encoding the rendered tagged frames into encoded frames; and
transmitting to the client the encoded frames and the metadata to cause the client to calculate input latency associated with the input event based on the metadata.
2 . The method of claim 1 , further comprising:
adjusting the tagging scheme based on rendering speed of rendering the frames into the rendered tagged frames.
3 . The method of claim 2 , wherein:
the rotation pattern includes a number of colors for tagging the frames; and
adjusting the tagging scheme based on the rendering speed of rendering the frames into the rendered tagged frames includes increasing or decreasing the number of colors for tagging the frames based on the rendering speed.
4 . The method of claim 2 , wherein:
the rotation pattern includes a number of pixels for tagging the frames; and
adjusting the tagging scheme based on the rendering speed of rendering the frames into the rendered tagged frames includes using different number of pixels for tagging the frames based on the rendering speed.
5 . The method of claim 1 , wherein tagging the frames corresponding to the TV UI includes:
drawing the at least one pixel in a predefined area in each of the frames.
6 . The method of claim 5 , wherein the predefined area is off-screen when the TV UI is displayed at the client.
7 . The method of claim 1 , wherein the input event forwarded by the client corresponds to an input received by the client and indicates an input time of the input received at the client.
8 . The method of claim 1 , wherein number of tags in the rotation pattern is determined based on expected time between two consecutive input events and rendering speed of rendering the frames.
9 . The method of claim 1 , wherein:
the rotation pattern specifies rotating among a first tag and a second tag responsive to receiving the input events; and
tagging the frames corresponding to the TV UI according to the first tagging scheme includes tagging each of the frames using the second tag in response to receiving the input event, different from the first tag used for tagging a previous screen responsive to receiving a previous input event.
10 . The method of claim 1 , wherein generating the metadata according to the rotation pattern includes:
identifying a tag change from a previous rendered frame in a first rendered frame of the rendered frames; and
generating the metadata that associate the input event with the first rendered frame.
11 . The method of claim 1 , wherein the metadata is included in a header of a first encoded frame of the encoded frames.
12 . The method of claim 1 , wherein transmitting to the client the encoded frames and the metadata to cause the client to calculate the input latency associated with the input event based on the metadata includes:
causing the client to decode the encoded frames and extract the metadata; and
causing the client to calculate the input latency based on current time and the metadata.
13 . A server comprising:
one or more processors;
a non-transitory memory;
a network interface; and
one or more programs, stored in the non-transitory memory, which, when executed by the one or more processors, cause the server to:
receive an input event from a client requesting a TV user interface (UI);
tag frames corresponding to the TV UI according to a tagging scheme prior to rendering the frames into rendered tagged frames, wherein the tagging scheme specifies a rotation pattern for tagging the frames responsive to receiving input events, and the tagging scheme uses at least one pixel following the rotation pattern;
generate metadata according to the rotation pattern to associate the input event with the rendered tagged frames prior to encoding the rendered tagged frames into encoded frames; and
transmit to the client the encoded frames and the metadata to cause the client to calculate input latency associated with the input event based on the metadata.
14 . The server of claim 13 , wherein the one or more programs, which, when executed by the one or more processors, further cause the server to:
adjust the tagging scheme based on rendering speed of rendering the frames into the rendered tagged frames.
15 . The server of claim 14 , wherein:
the rotation pattern includes a number of colors for tagging the frames; and
adjusting the tagging scheme based on the rendering speed of rendering the frames into the rendered tagged frames includes increasing or decreasing the number of colors for tagging the frames based on the rendering speed.
16 . The server of claim 14 , wherein:
the rotation pattern includes a number of pixels for tagging the frames; and
adjusting the tagging scheme based on the rendering speed of rendering the frames into the rendered tagged frames includes using different number of pixels for tagging the frames based on the rendering speed.
17 . The server of claim 13 , wherein tagging the frames corresponding to the TV UI includes:
drawing the at least one pixel in a predefined area in each of the frames.
18 . A non-transitory memory storing one or more programs, which, when executed by one or more processors of one or more servers, cause the one or more servers to:
receive an input event from a client requesting a TV user interface (UI);
tag frames corresponding to the TV UI according to a tagging scheme prior to rendering the frames into rendered tagged frames, wherein the tagging scheme specifies a rotation pattern for tagging the frames responsive to receiving input events, and the tagging scheme uses at least one pixel following the rotation pattern;
generate metadata according to the rotation pattern to associate the input event with the rendered tagged frames prior to encoding the rendered tagged frames into encoded frames; and
transmit to the client the encoded frames and the metadata to cause the client to calculate input latency associated with the input event based on the metadata.