Accounting for latency in content streams
There has been provided a method of modifying a content stream provided to a user electronic device, the method comprising receiving device data from the user electronic device, processing the device data to determine whether one or more network conditions have been met, and modifying the content stream and/or one or more settings associated with either the content stream and or a game server. There has also provided a method of rendering a content stream provided by a sender, the method comprising, sending device data, receiving a modified content stream, and rendering the modified content stream.
1 . A computer-implemented method comprising:
receiving a particular user input to a user electronic device after a time window for modifying a content stream based on user inputs had ended;
determining a presence of network latency for the user electronic device when the particular user input was received; and
in response to determining a presence of network latency for the user electronic device when the particular user input was received, modifying the content stream based on the particular user input that was received after the time window for modifying the content stream based on user inputs had ended.
2 . The computer-implemented method of claim 1 , wherein modifying the content stream based on the particular user input comprises portioning the content stream into a plurality of frames.
3 . The computer-implemented method of claim 2 , wherein portioning the content stream into a plurality of frames comprises:
identifying one or more dynamic elements and one or more static elements within each frame of the plurality of frames; and
for each frame, updating the one or more dynamic elements.
4 . The computer-implemented method of claim 3 , wherein for each frame, updating the one or more dynamic elements:
maintaining the one or more static elements from a previous frame.
5 . The computer-implemented method of claim 3 , wherein the state of the one or more static or the one or more dynamic elements can switch between the plurality of frames.
6 . The computer-implemented method of claim 5 , wherein a dynamic element of the one or more dynamic elements in a first frame of the content stream is a static element in a second frame of the content stream.
7 . The computer-implemented method of claim 1 , comprising:
modifying the content stream when no network latency was determined.
8 . The computer-implemented method of claim 1 , wherein modifying the content stream based on the particular user input comprises modifying a user input window r.
9 . The computer-implemented method of claim 1 , wherein the modifying the content stream based on the particular user input comprises adjusting a refresh rate of the content stream, wherein the refresh rate is indicative of a number of image frames of the content stream that are displayed per second.
10 . The computer-implemented method of claim 1 , wherein modifying the content stream based on the particular user input comprises replacing one or more elements within the content stream with a lower level of detail (LOD) version of the one or more elements.
11 . The computer-implemented method of claim 1 , wherein modifying the content stream based on the particular user input comprises:
portioning the content stream into a first and second portion, and wherein providing the content stream; and
providing the first portion to the user electronic device whilst the second portion is being buffered.
12 . The computer-implemented method of claim 1 , wherein modifying the content stream based on the particular user input comprises a combination of: (i) portioning the content stream into a plurality of frames, (ii) modifying a user input window r, (iii) adjusting a refresh rate of the content stream, (iv) replacing one or more elements within the content stream with a lower level of detail (LOD) version of the elements, and (v) portioning the content stream into a first and second portion.
13 . A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving a particular user input to a user electronic device after a time window for modifying a content stream based on user inputs had ended;
determining a presence of network latency for the user electronic device when the particular user input was received; and
in response to determining a presence of network latency for the user electronic device when the particular user input was received, modifying the content stream based on the particular user input that was received after the time window for modifying the content stream based on user inputs had ended.
14 . The system of claim 13 , wherein modifying the content stream based on the particular user input comprises portioning the content stream into a plurality of frames.
15 . The system of claim 14 , wherein portioning the content stream into a plurality of frames comprises:
identifying one or more dynamic elements and one or more static elements within each frame of the plurality of frames; and
for each frame, updating the one or more dynamic elements.
16 . The system of claim 15 , wherein for each frame, updating the one or more dynamic elements:
maintaining the one or more static elements from a previous frame.
17 . One or more non-transitory computer readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving a particular user input to a user electronic device after a time window for modifying a content stream based on user inputs had ended;
determining a presence of network latency for the user electronic device when the particular user input was received; and
in response to determining the presence of network latency for the user electronic device when the particular user input was received, modifying the content stream based on the particular user input that was received after the time window for modifying the content stream based on user inputs had ended.
18 . The one or more non transitory computer readable storage media of claim 17 , wherein modifying the content stream based on the particular user input comprises portioning the content stream into a plurality of frames.
19 . The one or more non transitory computer readable storage media of claim 18 , wherein portioning the content stream into a plurality of frames comprises:
identifying one or more dynamic elements and one or more static elements within each frame of the plurality of frames; and
for each frame, updating the one or more dynamic elements.
20 . The one or more non transitory computer readable storage media of claim 19 , wherein for each frame, updating the one or more dynamic elements:
maintaining the one or more static elements from a previous frame.