Method and system to reduce latency in cloud gaming
View Patent ↗The invention provides a computer-implemented method of reducing latency in a cloud gaming system, the cloud gaming system comprising a local user device and a connected remote server, the method comprising: receiving a user input at the user device, wherein the user input initiates a gameplay event, the gameplay event comprising a visual output and one or both of an audio output and a haptic output, each to be outputted locally at the local user device; processing graphical data remotely at a processing unit of the server to provide the visual output of the gameplay event; and processing one or both of audio data and haptic data locally at a processing unit of the user device to provide the one or both of the audio output and the haptic output of the gameplay event respectively.
1 . A computer-implemented method comprising:
determining, by a local user device, that a user is likely to initiate a gameplay event that includes (a) a visual output and (b)(i) an audio output or (b)(ii) a haptic output, while the local user device streams a game over a network from a cloud-based gaming service;
in response to determining that the user is likely to initiate the gameplay event and before the user has actually initiated the gameplay event, storing audio data associated with the audio output or haptic data associated with the haptic output at the local user device;
receiving a user input at the local user device that initiates the gameplay event;
in response to receiving the user input at the local user device that initiates the gameplay event:
transmitting a request for graphical data associated with the visual output of the gameplay event over the network to the cloud-based gaming service, then receiving the graphical data associated with the visual output of the gameplay event from the cloud-based gaming service over the network, then providing, for output, the visual output of the gameplay event using the graphical data; and
providing the audio output of the gameplay event using the stored audio data or the haptic output of the gameplay event using the haptic data.
2 . The computer-implemented method of claim 1 , wherein the audio output or the haptic output are provided for output independent of receiving the graphical data.
3 . The computer-implemented method of claim 1 , wherein the audio output or the haptic output are provided for output before the graphical data is received.
4 . The computer-implemented method of claim 1 , wherein the audio output or the haptic output are provided for output before the visual output is provided for output.
5 . The computer-implemented method of claim 1 , wherein storing audio data associated with the audio output or haptic data associated with the haptic at the local user device comprises:
before the user has actually initiated the gameplay event, transmitting a request over the network to the cloud-based gaming service for the audio data associated with the audio output of the gameplay event or for the haptic data that is associated with the haptic output of the gameplay event; and
receiving, over the network from the cloud-based gaming service, the audio data associated with the audio output of the gameplay event or for the haptic data that is associated with the haptic output of the gameplay event.
6 . The computer-implemented method of claim 1 , wherein determining that a user is likely to initiate a gameplay event that includes a) a visual output and (b)(i) an audio output or (b)(ii) a haptic output, while the local user device streams a game over a network from a cloud-based gaming service comprises determining a probability that the user is likely to initiate the gameplay event while the local user device streams a game over a network from a cloud-based gaming service.
7 . The computer-implemented method of claim 6 , wherein the probability is determined based on at least one of prior user inputs from the local user device, prior data associated with the gameplay event, prior data relating to a current state of the gameplay event, or data associated with a history of the gameplay event.
8 . The computer-implemented method of claim 6 , wherein determining a probability that the user is likely to initiate the gameplay event while the local user device streams a game over a network from a cloud-based gaming service comprises determining the probability that the user is likely to initiate the gameplay event prior to receiving the user input at the local user device that initiates the gameplay event.
9 . A system comprising:
one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
determining, by a local user device, that a user is likely to initiate a gameplay event that includes (a) a visual output and (b)(i) an audio output or (b)(ii) a haptic output, while the local user device streams a game over a network from a cloud-based gaming service;
in response to determining that the user is likely to initiate the gameplay event and before the user has actually initiated the gameplay event, storing audio data associated with the audio output or haptic data associated with the haptic output at the local user device;
receiving a user input at the local user device that the gameplay event;
in response to receiving the user input at the local user device that initiates the gameplay event:
transmitting a request for graphical data associated with the visual output of the gameplay event over the network to the cloud-based gaming service, then receiving the graphical data associated with the visual output of the gameplay event from the cloud-based gaming service over the network, then providing, for output, the visual output of the gameplay event using the graphical data; and
providing the audio output of the gameplay event using the stored audio data or the haptic output of the gameplay event using the haptic data.
10 . The system of claim 9 , wherein the audio output or the haptic output are provided for output independent of receiving the graphical data.
11 . The system of claim 9 , wherein the audio output or the haptic output are provided for output before the graphical data is received.
12 . The system of claim 9 , wherein the audio output or the haptic output are provided for output before the visual output is provided for output.
13 . The system of claim 9 , wherein storing audio data associated with the audio output or haptic data associated with the haptic output at the local user device comprises:
before the user has actually initiated the gameplay event, transmitting a request over the network to the cloud-based gaming service for the audio data associated with the audio output of the gameplay event or for the haptic data that is associated with the haptic output of the gameplay event; and
receiving, over the network from the cloud-based gaming service, the audio data associated with the audio output of the gameplay event or for the haptic data that is associated with the haptic output of the gameplay event.
14 . The system of claim 9 , wherein determining that a user is likely to initiate a gameplay event that includes a) a visual output and (b)(i) an audio output or (b)(ii) a haptic output, while the local user device streams a game over a network from a cloud-based gaming service comprises determining a probability that the user is likely to initiate the gameplay event while the local user device streams a game over a network from a cloud-based gaming service.
15 . The system of claim 14 , wherein the probability is determined based on at least one of prior user inputs from the local user device, prior data associated with the gameplay event, prior data relating to a current state of the gameplay event, or data associated with a history of the gameplay event.
16 . The system of claim 14 , wherein determining a probability that the user is likely to initiate the gameplay event while the local user device streams a game over a network from a cloud-based gaming service comprises determining the probability that the user is likely to initiate the gameplay event prior to receiving the user input at the local user device that initiates the gameplay event.
17 . One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
determining, by a local user device, that a user is likely to initiate a gameplay event that includes (a) a visual output and (b)(i) an audio output or (b)(ii) a haptic output, while the local user device streams a game over a network from a cloud-based gaming service;
in response to determining that the user is likely to initiate the gameplay event and before the user has actually initiated the gameplay event, storing audio data associated with the audio output or haptic data associated with the haptic output at the local user device;
receiving a user input at the local user device that the gameplay event;
in response to receiving the user input at the local user device that initiates the gameplay event:
transmitting a request for graphical data associated with the visual output of the gameplay event over the network to the cloud-based gaming service, then receiving the graphical data associated with the visual output of the gameplay event from the cloud-based gaming service over the network, then providing, for output, the visual output of the gameplay event using the graphical data; and
providing the audio output of the gameplay event using the stored audio data or the haptic output of the gameplay event using the haptic data.
18 . The one or more non-transitory computer storage media of claim 17 , wherein the audio output or the haptic output are provided for output independent of receiving the graphical data.
19 . The one or more non-transitory computer storage media of claim 17 , wherein the audio output or the haptic output are provided for output before the graphical data is received.
20 . The one or more non-transitory computer storage media of claim 17 , wherein the audio output or the haptic output are provided for output before the visual output is provided for output.