Method and system for decoder-side intra mode derivation for block-based video coding

1. A method for encoding video data by an encoder, comprising:

for a plurality of candidates of intra prediction modes, determining a cost of using a respective candidate mode to predict samples in a template region adjacent to a current block in the video data, wherein determining the costs comprises:

in an initial stage, determining costs of using respective intra prediction modes from an initial set of candidate modes, wherein the initial set of candidate modes includes a planar mode and a DC mode,

determining that neither the planar nor the DC mode is the candidate mode having the lowest cost, and

only in response to a determination that neither the planar nor the DC mode is the candidate mode having the lowest cost, in at least one subsequent stage, selecting a subsequent set of candidate modes based on the candidate mode in the previous stage having the lowest cost;

deriving an intra prediction mode from the plurality of candidates of intra prediction modes based on their respective costs; and

predicting the samples in the current block with intra prediction using the derived intra prediction mode.

2. The method of claim 1 , wherein a cost, of the determined costs, is a measure of distortion between the template region and a prediction of the template region using the respective candidate of intra prediction mode.

3. The method of claim 1 , wherein a bitstream encoding the video data includes a flag indicating that decoder-side intra mode derivation is used for the current block.

4. The method of claim 1 , wherein:

in the initial stage, the modes in the initial set are separated by an initial interval; and

in the at least one subsequent stage, the modes in the subsequent set are separated by a subsequent interval smaller than the interval used in the previous stage.

5. The method of claim 1 , wherein the derived mode is included in a list of most probable modes.

6. The method of claim 5 , wherein an index is coded in a bitstream identifying the derived mode from the list of most probable modes.

7. The method of claim 1 , wherein prediction residuals for the samples in the current block are coded in a bitstream using a transform coefficient scanning order, and wherein the selection of the transform coefficient scanning order is independent of the derived mode.

8. The method of claim 7 , wherein the transform coefficient scanning order is a predetermined scanning order.

9. The method of claim 7 , wherein the transform coefficient scanning order is based on intra modes of spatial neighbors of the current block.

10. An encoder system for encoding video data comprising: a processor configured to perform at least:

for a plurality of candidates of intra prediction modes, determining a cost of using a respective candidate mode to predict a template region adjacent to a current block in the video data, wherein determining the costs comprises:

in an initial stage, determining costs of using respective intra prediction modes from an initial set of candidate modes, wherein the initial set of candidate modes includes a planar mode and a DC mode, and

only in response to a determination that neither the planar nor the DC mode is the candidate mode having the lowest cost, in at least one subsequent stage, selecting a subsequent set of candidate modes based on the candidate mode in the previous stage having the lowest cost;

deriving an intra prediction mode from the plurality of candidates of intra prediction modes based on their respective costs; and

predicting the samples in the current block with intra prediction using the derived intra prediction mode.

11. A method comprising:

for a plurality of candidates of intra prediction modes, determining a cost of using a respective candidate mode to predict samples in a template region adjacent to a current block in a video, wherein determining the costs comprises:

in an initial stage, determining costs of using respective intra prediction modes from an initial set of candidate modes, and

in at least one subsequent stage, selecting a subsequent set of candidate modes based on the candidate mode in the previous stage having the lowest cost;

deriving an intra prediction mode from the plurality of candidates of intra prediction modes based on their respective costs; and

predicting the samples in the current block with intra prediction using the derived intra prediction mode;

wherein prediction residuals for the samples in the current block are coded in a bitstream using a transform coefficient scanning order, and wherein the selection of the transform coefficient scanning order is independent of the derived mode.

12. The method of claim 11 , wherein the initial set of candidate modes includes a planar mode and a DC mode, and wherein the subsequent set of candidate modes is selected only in response to a determination that neither the planar nor the DC mode is the candidate mode having the lowest cost.

13. The system of claim 10 , wherein determining a cost of a candidate of intra prediction mode comprises:

using the respective candidate of intra prediction mode to generate a prediction of the template region from a set of reconstructed reference samples; and

measuring distortion between the template region and the prediction of the template region.

14. The system of claim 10 , wherein a bitstream encoding the video data includes a flag indicating that decoder-side intra mode derivation is used for the current block.

15. The system of claim 10 , wherein at least some blocks in the video data are predicted using a predetermined set of explicitly-signaled intra modes, wherein the candidates of intra prediction modes have a finer granularity than the predetermined set of explicitly signaled intra modes.

16. The method of claim 11 , wherein the transform coefficient scanning order is a predetermined scanning order.

17. The method of claim 11 , wherein the transform coefficient scanning order is based on intra modes of spatial neighbors of the current block.

18. The method of claim 11 , performed by a video encoder.

19. The method of claim 11 , performed by a video decoder.

20. A method for decoding video data by a decoder, comprising:

for a plurality of candidates of intra prediction modes, determining a cost of using a respective candidate mode to predict samples in a template region adjacent to a current block in the video data, wherein determining the costs comprises:

in an initial stage, determining costs of using respective intra prediction modes from an initial set of candidate modes, wherein the initial set of candidate modes includes a planar mode and a DC mode,

determining that neither the planar nor the DC mode is the candidate mode having the lowest cost, and

only in response to a determination that neither the planar nor the DC mode is the candidate mode having the lowest cost, in at least one subsequent stage, selecting a subsequent set of candidate modes based on the candidate mode in the previous stage having the lowest cost;

deriving an intra prediction mode from the plurality of candidates of intra prediction modes based on their respective costs; and

predicting the samples in the current block with intra prediction using the derived intra prediction mode.

21. A decoder system for decoding video data comprising:

a processor configured to perform at least:

for a plurality of candidates of intra prediction modes, determining a cost of using a respective candidate mode to predict samples in a template region adjacent to a current block in the video data, wherein determining the costs comprises:

in an initial stage, determining costs of using respective intra prediction modes from an initial set of candidate modes, wherein the initial set of candidate modes includes a planar mode and a DC mode, and

only in response to a determination that neither the planar nor the DC mode is the candidate mode having the lowest cost, in at least one subsequent stage, selecting a subsequent set of candidate modes based on the candidate mode in the previous stage having the lowest cost;

deriving an intra prediction mode from the plurality of candidates of intra prediction modes based on their respective costs; and

predicting the samples in the current block with intra prediction using the derived intra prediction mode.