Methods for compressing video for streaming video game content to remote clients
Computer-implemented methods for hosting online video games are provided. One method includes generating a plurality of video frames. The method includes initiating a sending of each one of the plurality of video frames to a client. Each of the video frames that is sent is compressed. The method includes stopping the compression and sending of video frames when one of the plurality of video frames is taking longer than a frame time to compress and send. A frame time is defined as one over a frame rate, and wherein stopping the compression of video frames includes ignoring said video frames by an encoder. The method includes continuing to compress and send audio data to the client when one or more of said plurality of video frames are not sent to the client.
1. A computer-implemented method, comprising:
receiving, from an interface provided by a server, a request from a client of a user for playing a video game;
determining, periodically by the server, a data rate available for a channel for communicating with the client, the data rate for the channel varies over time;
identifying, by the server, a target data rate for transmitting video frames for display on the client;
executing, at the server, the video game responsive to control inputs received from the client and said control inputs are used to responsively generate a sequence of video frames for a video output of the video game, the sequence of video frames for the video output of the video game are continuously updated based on said control inputs received from the client device while interacting with the video game, and such interaction with the video game causes real-time generation of the sequence of video frames, responsive to the control inputs, that represent the video output of the video game; and
preparing to send a first video frame and a second video frame that immediately follows the first video frame within the sequence of video frames to the client, the preparing is for said real-time generation of said sequence of video frames;
wherein, before sending the second video frame, the second video frame is ignored by an encoder associated with the server and not sent to the client in response to detecting by the server that the data rate to said client during said real-time generation is below the target data rate after sending the first video frame encoded by the encoder;
wherein audio data associated with the second video frame continues to be sent even when the second video frame is not sent.
2. The computer-implemented method of claim 1 , wherein sending the first video frame includes compressing by the encoder the first video frame and then transmitting the first video frame to the client.
3. The computer-implemented method of claim 1 , further comprising,
not transmitting one or more frames immediately following the second video frame within the sequence of video frames if the data rate remains below the target data rate.
4. The computer-implemented method of claim 1 , wherein the data rate being below the target data rate is based on a video frame taking longer than a frame time to transmit, the frame time being one divided by a frame rate.
5. The computer-implemented method of claim 1 , wherein the client, when not receiving the second video frame, is configured to display the first video frame during a frame time during which the second video frame would have been displayed.
6. The computer-implemented method of claim 5 , wherein displaying the first video frame during the frame time during which the second video frame would have been displayed reduces a frame at the client.
7. The computer-implemented method of claim 1 , wherein not sending the second video frame causes the client to have a reduction in a frame rate.
8. The computer-implemented method of claim 1 , wherein the client has a reduced frame rate by displaying the first video frame during a time when the second video frame or any additional video frames following the second video frame are not sent to the client due to the data rate falling below the target data rate.
9. The computer-implemented method of claim 1 , wherein said audio that is transmitted is compressed and the encoder does not drop audio data when video frames are not compressed.
10. Non-transitory computer readable media having program instructions for processing a method, comprising:
program instructions for receiving, from an interface provided by a server, a request from a client of a user for playing a video game;
program instructions for determining, periodically by the server, a data rate available for a channel for communicating with the client, the data rate for the channel varies over time;
program instructions for identifying, by the server, a target data rate for transmitting video frames for display on the client;
program instructions for executing, at the server, the video game responsive to control inputs received from the client and said control inputs are used to responsively generate a sequence of video frames for a video output of the video game, the sequence of video frames for the video output of the video game are continuously updated based on said control inputs received from the client device while interacting with the video game, and such interaction with the video game causes real-time generation of the sequence of video frames, responsive to the control inputs, that represent the video output of the video game; and
program instructions for preparing to send a first video frame and a second video frame that immediately follows the first video frame within the sequence of video frames to the client, the preparing is for said real-time generation of said sequence of video frames;
wherein, before sending the second video frame, the second video frame is ignored by an encoder associated with the server and not sent to the client in response to detecting by the server that the data rate to said client during said real-time generation is below the target data rate after sending the first video frame encoded by the encoder;
wherein audio data associated with the second video frame continues to be sent even when the second video frame is not sent.
11. The Non-transitory computer readable media of claim 10 , wherein sending the first video frame includes compressing by the encoder the first video frame and then transmitting the first video frame to the client.
12. The Non-transitory computer readable media of claim 10 , further comprising,
program instructions for not transmitting one or more frames immediately following the second video frame within the sequence of video frames if the data rate remains below the target data rate.
13. The Non-transitory computer readable media of claim 10 , wherein the data rate being below the target data rate is based on a video frame taking longer than a frame time to transmit, the frame time being one divided by a frame rate.
14. The Non-transitory computer readable media of claim 10 , wherein the client, when not receiving the second video frame, is configured to display the first video frame during a frame time during which the second video frame would have been displayed.
15. The Non-transitory computer readable media of claim 14 , wherein displaying the first video frame during the frame time during which the second video frame would have been displayed reduces a frame at the client.
16. The Non-transitory computer readable media of claim 10 , wherein not sending the second video frame causes the client to have a reduction in a frame rate.
17. The Non-transitory computer readable media of claim 10 , wherein the client has a reduced frame rate by displaying the first video frame during a time when the second video frame or any additional video frames following the second video frame are not sent to the client due to the data rate falling below the target data rate.
18. The Non-transitory computer readable media of claim 10 , wherein said audio that is transmitted is compressed and the compressor does not drop audio data when video frames are not compressed.
19. The method of claim 1 , wherein the data rate of the channel varies over time during session of playing said video game, wherein a plurality of frames are sent from the server to the client before determining that the second video frame is to be ignored, and said determining that said second video frame is to be ignored is responsive to determining during the session that the data rate is below the target data rate after the first video frame is sent.
20. The Non-transitory computer readable media of claim 10 , wherein the data rate of the channel varies over time during session of playing said video game, wherein a plurality of frames are sent from the server to the client before determining that the second video frame is to be ignored, and said determining that said second video frame is to be ignored is responsive to determining during the session that the data rate is below the target data rate after the first video frame is sent.