Reducing latency in video encoding and decoding

1. In a computing system that implements a video encoder, a method comprising:

setting a constraint on delay in terms of a time measure;

determining a frame rate for encoding;

based on the constraint on delay and the frame rate, setting a constraint on frame reordering latency consistent with inter-frame dependencies between plural frames of a video sequence;

setting one or more syntax elements that indicate the constraint on frame reordering latency, wherein the constraint on frame reordering latency is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in output order but follow that frame in coded order;

with the video encoder, encoding the plural frames to produce encoded data, wherein the encoding uses the inter-frame dependencies that are consistent with the constraint on frame reordering latency; and

outputting the one or more syntax elements, thereby facilitating determination of when reconstructed frames are ready for output in terms of output order of the plural frames.

2. The method of claim 1 , wherein the constraint on delay is set as part of negotiation to trade off responsiveness with ability to smooth out short-term fluctuations in bit rate, encoding complexity, and/or network bandwidth.

3. The method of claim 1 , further comprising:

selecting, from multiple available encoding modes, a low-latency encoding mode for real-time communication, wherein the constraint on delay is set as part of the low-latency encoding mode.

4. The method of claim 1 , wherein the time measure is independent of frame rate.

5. The method of claim 1 , further comprising:

before the encoding, receiving the plural frames of the video sequence; and

after the encoding, outputting the encoded data for storage or transmission.

6. The method of claim 1 , wherein a value of the one or more syntax elements indicates an integer count for the constraint on frame reordering latency relative to a maximum size of frame memory for reordering, the maximum size of frame memory for reordering being indicated with a different syntax element.

7. The method of claim 6 , wherein the constraint on frame reordering latency can be determined as a maximum count for the maximum size of frame memory for reordering plus the integer count for the constraint on frame reordering latency minus 1.

8. A computing system comprising one or more processing units and memory, wherein the computing system is configured to perform operations comprising:

as part of negotiation to trade off responsiveness with ability to smooth out short-term fluctuations in bit rate, encoding complexity, and/or network bandwidth, setting a constraint on frame reordering latency consistent with inter-frame dependencies between plural frames of a video sequence;

setting one or more syntax elements that indicate the constraint on frame reordering latency consistent, wherein the constraint on frame reordering latency is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in output order but follow that frame in coded order;

encoding the plural frames to produce encoded data, wherein the encoding uses the inter-frame dependencies that are consistent with the constraint on frame reordering latency; and

outputting the one or more syntax elements, thereby facilitating determination of when reconstructed frames are ready for output in terms of output order of the plural frames.

9. The computing system of claim 8 , wherein the operations further comprise, as part of the negotiation:

setting a constraint on delay in terms of a time measure; and

determining a frame rate for encoding, wherein the constraint on frame reordering latency is based on the constraint on delay and the frame rate.

10. The computing system of claim 8 , wherein the operations further comprise:

selecting, from multiple available encoding modes, a low-latency encoding mode for real-time communication, wherein the constraint on frame reordering latency is set as part of the low-latency encoding mode.

11. The computing system of claim 8 , wherein the negotiation is between a transmitter-side tool that includes a video encoder and a receiver-side tool that includes a video decoder.

12. The computing system of claim 8 , wherein the operations further comprise:

before the encoding, receiving the plural frames of the video sequence; and

after the encoding, outputting the encoded data for storage or transmission.

13. The computing system of claim 8 , wherein a value of the one or more syntax elements indicates an integer count for the constraint on frame reordering latency relative to a maximum size of frame memory for reordering, the maximum size of frame memory for reordering being indicated with a different syntax element.

14. The computing system of claim 13 , wherein the constraint on frame reordering latency can be determined as a maximum count for the maximum size of frame memory for reordering plus the integer count for the constraint on frame reordering latency minus 1.

15. One or more computer-readable media storing computer-executable instructions for causing a computing system, when programmed thereby, to perform operations, the one or more computer-readable media being selected only from the group consisting of non-volatile memory and storage device, the operations comprising:

selecting, from multiple available encoding modes, a low-latency encoding mode for real-time communication;

as part of the low-latency encoding mode, setting a constraint on frame reordering latency consistent with inter-frame dependencies between plural frames of a video sequence;

setting one or more syntax elements that indicate the constraint on frame reordering latency, wherein the constraint on frame reordering latency is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in output order but follow that frame in coded order;

encoding the plural frames to produce encoded data, wherein the encoding uses the inter-frame dependencies that are consistent with the constraint on frame reordering latency; and

outputting the one or more syntax elements, thereby facilitating determination of when reconstructed frames are ready for output in terms of output order of the plural frames.

16. The one or more computer-readable media of claim 15 , wherein the multiple available encoding modes further include a regular encoding mode for media playback from a file or stream.

17. The one or more computer-readable media of claim 15 , wherein the operations further comprise, as part of the low-latency encoding mode:

setting a constraint on delay in terms of a time measure; and

determining a frame rate for encoding, wherein the constraint on frame reordering latency is based on the constraint on delay and the frame rate.

18. The one or more computer-readable media of claim 15 , wherein the constraint on frame reordering latency is set as part of negotiation to trade off responsiveness with ability to smooth out short-term fluctuations in bit rate, encoding complexity, and/or network bandwidth.

19. The one or more computer-readable media of claim 15 , wherein a value of the one or more syntax elements indicates an integer count for the constraint on frame reordering latency relative to a maximum size of frame memory for reordering, the maximum size of frame memory for reordering being indicated with a different syntax element.

20. The one or more computer-readable media of claim 19 , wherein the constraint on frame reordering latency can be determined as a maximum count for the maximum size of frame memory for reordering plus the integer count for the constraint on frame reordering latency minus 1.