Server-based encoding of adjustable frame rate content
Systems, apparatuses, and methods for encoding bitstreams of uniquely rendered video frames with variable frame rates are disclosed. A rendering unit and an encoder in a server are coupled via a network to a client with a decoder. The rendering unit dynamically adjusts the frame rate of uniquely rendered frames. Depending on the operating mode, the rendering unit conveys a constant frame rate to the encoder by repeating some frames or the rendering unit conveys a variable frame rate to the encoder by conveying only uniquely rendered frames to the encoder. Depending on the operating mode, the encoder conveys a constant frame rate bitstream to the decoder by encoding repeated frames as skip frames, or the encoder conveys a variable frame rate bitstream to the decoder by dropping repeated frames from the bitstream.
1. A system comprising:
a rendering unit configured to dynamically adjust a rate at which video frames are rendered based on an amount of activity in a scene being rendered; and
an encoder configured to encode rendered video frames;
wherein the encoder is configured to:
detect a reduction of a frame rate of rendered video frames;
responsive to detecting said reduction:
determine whether to create an encoded bitstream with a fixed frame or a variable frame rate based on capabilities of a decoder;
encode the rendered video frames and one or more skip frames to create an encoded bitstream with a fixed frame rate while operating in a first operating mode;
encode only the rendered video frames, without any skip frames, to create the encoded bitstream with a variable frame rate while operating in a second operating mode; and
convey the encoded bitstream to a decoder.
2. The system as recited in claim 1 , wherein the rendering unit is configured to:
reduce a frame rate of rendered video frames responsive to receiving an indication from an application indicating that an amount of activity in a scene is less than a threshold; and
repeat one or more video frames to create a sequence of frames of a fixed frame rate responsive to reducing the frame rate of uniquely rendered video frames.
3. The system as recited in claim 2 , wherein the rendering unit is configured to convey, to the encoder, the sequence of frames at the fixed frame rate.
4. The system as recited in claim 1 , wherein the rendering unit is configured to reduce a frame rate of uniquely rendered video frames responsive to receiving an indication from an application that a current scene can be rendered with a lower frame rate.
5. The system as recited in claim 4 , wherein the encoder is configured to reallocate a remainder of bits of a bit budget for encoding unique video frames to allow the uniquely rendered video frames to be encoded with increased fidelity responsive to detecting repeated video frames in the sequence of frames provided by the rendering unit.
6. The system as recited in claim 1 , further comprising a decoder, wherein the decoder is configured to perform a post decode comparison to determine if a current decoded frame is the same as a previous decoded frame.
7. The system as recited in claim 1 , wherein the encoder is configured to determine an operating mode based on information received in a request for rendered content generated by the decoder and based on whether the decoder can process frames at a variable frame rate without causing synchronization issues.
8. A method comprising:
dynamically adjusting, by a rendering unit, a rate at which video frames are rendered based on an amount of activity in a scene being rendered;
detecting, by an encoder, a reduction in a frame rate of rendered video frames;
responsive to detecting said reduction:
determine whether to create an encoded bitstream with a fixed frame or a variable frame rate based on capabilities of a decoder;
encoding the rendered video frames and one or more skip frames to create an encoded bitstream with a fixed frame rate while operating in a first operating mode;
encoding only the rendered video frames, without any skip frames, to create the encoded bitstream with a variable frame rate while operating in a second operating mode; and
conveying the encoded bitstream to a decoder.
9. The method as recited in claim 8 , further comprising:
reducing a frame rate of rendered video frames responsive to receiving an indication from an application indicating that an amount of activity in a scene is less than a threshold; and
repeating one or more video frames to create a sequence of frames of a fixed frame rate responsive to reducing the frame rate of uniquely rendered video frames.
10. The method as recited in claim 9 , wherein the method further comprising conveying, from a rendering unit to an encoder, the sequence of frames at the fixed frame rate.
11. The method as recited in claim 8 , further comprising the rendering unit reducing a frame rate of uniquely rendered video frames responsive to receiving an indication from an application that a current scene can be rendered with a lower frame rate.
12. The method as recited in claim 11 , further comprising reallocating a remainder of bits of a bit budget for encoding unique video frames to allow the uniquely rendered video frames to be encoded with increased fidelity responsive to detecting repeated video frames in the sequence of frames provided by the rendering unit.
13. The method as recited in claim 8 , further comprising sending an indication from the rendering unit to the encoder that the frame rate of rendered video frames has been reduced.
14. The method as recited in claim 8 , further comprising determining an operating mode based on information received in a request for rendered content generated by the decoder and based on whether the decoder can process frames at a variable frame rate without causing synchronization issues.
15. An apparatus comprising:
a rendering unit configured to dynamically adjust a rate at which video frames are rendered based on an amount of activity in a scene being rendered;
a pre-encoding analysis unit configured to detect repeated frames in a sequence of frames; and
an encoder coupled to the pre-encoding analysis unit;
wherein the encoder is configured to:
receive an indication from the pre-encoding analysis unit that a frame rate of rendered video frames has been reduced; and
responsive to receiving said indication:
determine whether to create an encoded bitstream with a fixed frame or a variable frame rate based on capabilities of a decoder;
encode the rendered video frames and one or more skip frames to create an encoded bitstream with a fixed frame rate while operating in a first operating mode;
encode only the rendered video frames, without any skip frames, to create the encoded bitstream with a variable frame rate while operating in a second operating mode; and
convey the encoded bitstream to a decoder.
16. The apparatus as recited in claim 15 , wherein the apparatus is further configured to:
reduce a frame rate of rendered video frames responsive to receiving an indication from an application indicating that an amount of activity in a scene is less than a threshold; and
repeat one or more video frames to create a sequence of frames of a fixed frame rate responsive to reducing the frame rate of uniquely rendered video frames.
17. The apparatus as recited in claim 16 , wherein the rendering unit is configured to convey, to the encoder, the sequence of frames at the fixed frame rate.
18. The apparatus as recited in claim 16 , wherein the rendering unit is configured to reduce a frame rate of uniquely rendered video frames responsive to receiving an indication from an application that a current scene can be rendered with a lower frame rate.
19. The apparatus as recited in claim 15 , wherein the encoder is configured to reallocate a remainder of bits of a bit budget for encoding unique video frames to allow the uniquely rendered video frames to be encoded with increased fidelity responsive to said indication.
20. The apparatus as recited in claim 15 , wherein the encoder is configured to determine an operating mode based on information received in a request for rendered content generated by the decoder and based on whether the decoder can process frames at a variable frame rate without causing synchronization issues.