Usage of templates for decoder-side intra mode derivation
Example implementations include a method, apparatus and computer-readable medium of video processing, including constructing, during a conversion between a current video block of a video and a bitstream of the video, at least one template set for the current video block from a plurality of sub-templates. The one or more sub-templates may be selected from a plurality of sub-templates including: a left sub-template, an above sub-template, a right-above sub-template, a left-below sub-template, and a left-above sub-template. The implementations further include deriving at least one intra-prediction mode (IPM) based on cost calculations. The implementations include determining, based on the at least one IPM, a final predictor of the current video block. The implementations include performing the conversion based on the final predictor.
1. A method of processing a video, wherein the video comprises a plurality of frames, each frame comprising blocks of samples, the method comprising, for at least a current block:
selecting, during a conversion between the current block of the video and a bitstream of the video, a first template for the current block;
determining a cost of using each of a plurality of candidate intra prediction modes (IPMs) to predict samples in a template region, wherein determining the cost of using each of the plurality of candidate IPMs to predict samples comprises determining one or more lines of template-reference samples neighboring the first template based on the first template;
selecting a derived IPM from the plurality of candidate IPMs based on the cost; and
predicting the samples in the current block with intra-prediction using the derived IPM; and
performing the conversion based on the predicting,
wherein selecting the template is dependent on coded information including at least one of: block size, color format, single or dual tree partitioning, color component, slice type or picture type.
2. The method of claim 1 , wherein the one or more lines of template-reference samples include a horizontal row above the first template and a vertical column to the left of the first template.
3. The method of claim 1 , wherein the one or more lines of template-reference samples includes N lines of parallel rows or columns, where N is an integer greater than 1.
4. The method of claim 3 , wherein predicting the samples in the current block with intra-prediction using the derived IPM comprises selecting one line of the N lines.
5. The method of claim 4 , wherein selecting the one line of the N lines comprises implicitly deriving the one line.
6. The method of claim 4 , wherein selecting the one line of the N lines is related to a syntax element presented in the bitstream.
7. The method of claim 4 , wherein the one line of the N lines is inherited from an index of a reference line for multiple reference line (MRL).
8. The method of claim 3 , further comprises down-sampling the one or more lines to derive a single line.
9. The method of claim 3 , wherein a number of lines of template-reference samples is based on the first template.
10. The method of claim 9 , wherein the number of lines of template-reference samples is based on coded information for the current block.
11. The method of claim 1 , wherein determining one or more lines of template-reference samples comprises substituting an unavailable template-reference sample with an available reference sample.
12. The method of claim 11 , wherein the available template-reference sample is a nearest available template-reference sample.
13. The method of claim 11 , wherein the available template-reference sample is a predefined value, or a value based on a defined formula.
14. The method of claim 13 , wherein the defined formula is based on a bit depth of a color component.
15. The method of claim 13 , wherein the defined formula is based on a distance from the available template-reference sample.
16. The method of claim 11 , wherein the available template-reference sample is generated based on a reference sample substitution process.
17. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
select, during a conversion between a current block of the video and a bitstream of the video, a first template for the current block;
determine a cost of using each of a plurality of candidate intra prediction modes (IPMs) to predict samples in a template region, wherein determining the cost of using each of the plurality of candidate IPMs to predict samples comprises determining one or more lines of template-reference samples neighboring the first template based on the first template;
select a derived IPM from the plurality of candidate IPMs based on the cost; and
predict the samples in the current block with intra-prediction using the derived IPM; and
perform the conversion based on the predicted samples,
wherein the instructions upon execution by the processor, cause the processor to select the template dependent on coded information including at least one of: block size, color format, single or dual tree partitioning, color component, slice type or picture type.
18. A non-transitory computer-readable storage medium storing instructions that cause a processor to:
select, during a conversion between a current block of a video and a bitstream of the video, a first template for the current block;
determine a cost of using each of a plurality of candidate intra prediction modes (IPMs) to predict samples in a template region, wherein determining the cost of using each of the plurality of candidate IPMs to predict samples comprises determining one or more lines of template-reference samples neighboring the first template based on the first template;
select a derived IPM from the plurality of candidate IPMs based on the cost;
predict the samples in the current block with intra-prediction using the derived IPM; and
perform the conversion based on the predicted samples,
wherein the instructions to select the template are dependent on coded information including at least one of: block size, color format, single or dual tree partitioning, color component, slice type or picture type.
19. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
selecting a first template for a current block of the video;
determining a cost of using each of a plurality of candidate intra prediction modes (IPMs) to predict samples in a template region, wherein determining the cost of using each of the plurality of candidate IPMs to predict samples comprises determining one or more lines of template-reference samples neighboring the first template based on the first template;
selecting a derived IPM from the plurality of candidate IPMs based on the cost;
predicting the samples in the current block with intra-prediction using the derived IPM; and
generating the bitstream based on the predicting,
wherein selecting the template is dependent on coded information including at least one of: block size, color format, single or dual tree partitioning, color component, slice type or picture type.