Reducing latency in video encoding and decoding

1. A computing system comprising one or more processing units and memory, wherein the computing system implements a video decoder configured to perform operations comprising:

receiving and parsing one or more syntax elements that indicate a constraint on frame reordering latency, wherein the constraint on frame reordering latency is expressed in terms of a maximum count of frames that can precede any frame of a video sequence in output order but follow that frame in coded order;

determining the constraint on frame reordering latency based upon the one or more syntax elements;

receiving encoded data for plural frames of the video sequence;

decoding at least some of the encoded data to reconstruct one of the plural frames; and

outputting the reconstructed frame, wherein the constraint is used to determine when the reconstructed frame is ready for output.

2. The computing system of claim 1 , wherein the operations further comprise:

using the constraint on frame reordering latency to determine when the reconstructed frame is ready for output in terms of output order of the plural frames of the video sequence.

3. The computing system of claim 2 , wherein the plural frames of the video sequence are organized according to a temporal hierarchy, wherein different syntax elements indicate different constraints on frame reordering latencies for different temporal layers of the temporal hierarchy, the operations further comprising selecting one of the different constraints on frame reordering latencies depending on temporal resolution of the output.

4. The computing system of claim 1 , wherein the one or more syntax elements and the encoded data are signaled as part of syntax for a coded video bitstream, the operations further comprising:

receiving and parsing a buffer size syntax element that indicates maximum size of a decoded picture buffer, wherein the buffer size syntax element is different than the one or more syntax elements that indicate the constraint on frame reordering latency.

5. The computing system of claim 1 , wherein the one or more syntax elements are signaled as part of a sequence parameter set or media metadata relating to the encoded data.

6. The computing system of claim 1 , wherein the operations further comprise:

receiving a flag that indicates presence or absence of the one or more syntax elements, wherein if the flag indicates the one or more syntax elements are absent then the constraint on frame reordering latency is undefined or has a default value.

7. The computing system of claim 1 , wherein one possible value of the one or more syntax elements indicates the constraint on frame reordering latency is undefined or has a default value, and wherein other possible values of the one or more syntax elements indicate an integer count for the constraint on frame reordering latency.

8. The computing system of claim 1 , wherein a value of the one or more syntax elements indicates an integer count for the constraint on frame reordering latency relative to a maximum size of frame memory for reordering, the maximum size of frame memory for reordering being indicated with a different syntax element.

9. The computing system of claim 8 , wherein the constraint on frame reordering latency can be determined as a maximum count for the maximum size of frame memory for reordering plus the integer count for the constraint on frame reordering latency minus 1.

10. A computing system comprising one or more processing units and memory, wherein the computing system implements a video encoder configured to perform operations comprising:

setting one or more syntax elements that indicate a constraint on frame reordering latency consistent with inter-frame dependencies between plural frames of a video sequence, wherein the constraint on frame reordering latency is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in output order but follow that frame in coded order;

encoding the plural frames to produce encoded data, wherein the encoding uses the inter-frame dependencies that are consistent with the constraint on frame reordering latency; and

outputting the one or more syntax elements, thereby facilitating determination of when reconstructed frames are ready for output in terms of output order of the plural frames.

11. The computing system of claim 10 , wherein the operations further comprise:

before the encoding, receiving the plural frames of the video sequence; and

after the encoding, outputting the encoded data for storage or transmission.

12. The computing system of claim 10 , wherein the one or more syntax elements and the encoded data are output as part of syntax for a coded video bitstream, the operations further comprising:

outputting a buffer size syntax element that indicates maximum size of a decoded picture buffer, wherein the buffer size syntax element is different than the one or more syntax elements that indicate the constraint on frame reordering latency.

13. The computing system of claim 10 , wherein the one or more syntax elements are output as part of a sequence parameter set or media metadata relating to encoded data.

14. The computing system of claim 10 , wherein the operations further comprise:

outputting a flag that indicates presence or absence of the one or more syntax elements, wherein if the flag indicates the one or more syntax elements are absent then the constraint on frame reordering latency is undefined or has a default value.

15. The computing system of claim 10 , wherein one possible value of the one or more syntax elements indicates the constraint on frame reordering latency is undefined or has a default value, and wherein other possible values of the one or more syntax elements indicate an integer count for the constraint on frame reordering latency.

16. The computing system of claim 10 , wherein a value of the one or more syntax elements indicate an integer count for the constraint on frame reordering latency relative to a maximum size of frame memory for reordering, the maximum size of frame memory for reordering being indicated with a different syntax element.

17. The computing system of claim 16 , wherein the constraint on frame reordering latency can be determined as a maximum count for the maximum size of frame memory for reordering plus the integer count for the constraint on frame reordering latency minus 1.

18. One or more computer-readable media storing computer-executable instructions for causing a computing system, when programmed thereby, to perform operations, the one or more computer-readable media being selected only from the group consisting of non-volatile memory and storage device, the operations comprising:

setting one or more syntax elements that indicate a constraint on frame reordering latency consistent with inter-frame dependencies between plural frames of a video sequence, wherein the constraint on frame reordering latency is expressed in terms of a maximum count of frames that can precede any frame of the video sequence in output order but follow that frame in coded order;

encoding the plural frames to produce encoded data, wherein the encoding uses the inter-frame dependencies that are consistent with the constraint on frame reordering latency; and

outputting the one or more syntax elements, thereby facilitating determination of when reconstructed frames are ready for output in terms of output order of the plural frames.

19. The one or more computer-readable media of claim 18 , wherein the one or more syntax elements and the encoded data are output as part of syntax for a coded video bitstream, the operations further comprising:

outputting a buffer size syntax element that indicates maximum size of a decoded picture buffer, wherein the buffer size syntax element is different than the one or more syntax elements that indicate the constraint on frame reordering latency.

20. The one or more computer-readable media of claim 18 , wherein a value of the one or more syntax elements indicate an integer count for the constraint on frame reordering latency relative to a maximum size of frame memory for reordering, the maximum size of frame memory for reordering being indicated with a different syntax element.