Method and apparatus of intra prediction in image and video processing

1. A method of processing video data by intra prediction in a video coding system, comprising:

receiving input data associated with a current block in a current picture, wherein the current block is a non-square block and is coded or to be coded by intra prediction, a maximum of a width and a height of the current block is a longer side of the current block, and a minimum of the width and the height of the current block is a shorter side of the current block;

determining a current intra prediction mode to derive an intra predictor for the current block based on the width, the height, or both of the current block, wherein the current intra prediction mode is selected from a current mode set comprising a first set of angular intra prediction modes for the longer side of the current block and the second set of angular intra prediction modes for the shorter side of the current block, wherein a number of the first set of the angular intra prediction modes is greater than a number of a second set of angular intra prediction modes for the shorter side of the current block; and

encoding or decoding the current block according to the current intra prediction mode.

2. The method of claim 1 , wherein the first set of angular intra prediction modes is determined depending on a length of the longer side of the current block, and the second set of angular intra prediction modes is determined depending on a length of the shorter side of the current block.

3. The method of claim 1 , wherein two or more thresholds are used to compare with the longer side or the shorter side of the current block for determining the first set of angular intra prediction modes for the longer side of the current block.

4. The method of claim 1 , wherein the first set of angular intra prediction modes for the longer side of the current block is determined depending on a length of the longer side of the current block.

5. The method of claim 1 , wherein the first set of angular intra prediction modes for the shorter side of the current block is determined depending on a length of the longer side of the current block.

6. The method of claim 1 , wherein said encoding or decoding the current block according to the current intra prediction mode further comprises:

deriving most probable modes (MPMs) for the current block only if a number of intra prediction modes in the current mode set is greater than a threshold; and

if the number of intra prediction modes in the current mode set is greater than the threshold, the current intra prediction mode is coded using an MPM flag if the current intra prediction mode is equal to one of the MPMs or coded using fixed length coding if the current intra prediction mode is not equal to any of the MPMs.

7. The method of claim 6 , wherein the current intra prediction mode is coded using fixed length coding, truncated binary coding, exponential-golomb coding, or variable length coding if the number of intra prediction modes in the current mode set is not greater than the threshold.

8. The method of claim 7 , wherein the fixed length coding is applied to code the current intra prediction mode if the number of intra prediction modes in the current mode set is a power of two, and the variable length coding is applied to code the current intra prediction mode if the number of intra prediction modes in the current mode set is not a power of two.

9. The method of claim 1 , wherein said encoding or decoding the current block according to the current intra prediction mode further comprises:

selecting a coding method for encoding or decoding the current intra prediction mode according to a number of intra prediction mode in the current mode set.

10. The method of claim 1 , further comprising:

deriving a number of most probable modes (MPMs) in an MPM set from a number of intra prediction modes in the current mode set, wherein the number of MPMs in the MPM set varies with the number of intra prediction modes in the current mode set;

determining the MPMs for the current block; and

encoding or decoding the current intra prediction mode using an MPM flag if the current intra prediction mode is equal to one of the MPMs, and encoding or decoding the current intra prediction mode using fixed length coding if the current intra prediction mode is not equal to any of the MPMs.

11. The method of claim 10 , wherein the number of MPMs in the MPM set is derived by deducting two to the power of n from the number of intra prediction modes in the current mode set, n is an integer greater than 1, and the number of intra prediction modes in the current mode set is greater than two to the power of n and is less than two to the power of n+1.

12. An apparatus of processing video data by intra prediction in a video coding system, the apparatus comprising one or more electronic circuits configured for:

receiving input data associated with a current block in a current picture, wherein the current block is a non-square block and is coded or to be coded by intra prediction, a maximum of a width and a height of the current block is a longer side of the current block, and a minimum of the width and the height of the current block is a shorter side of the current block;

determining a current intra prediction mode to derive an intra predictor for the current block based on the width, the height, or both of the current block, wherein the current intra prediction mode is selected from a current mode set comprising a first set of angular intra prediction modes for the longer side of the current block and the second set of angular intra prediction modes for the shorter side of the current block, wherein a number of the first set of the angular intra prediction modes is greater than a number of a second set of angular intra prediction modes for the shorter side of the current block; and

encoding or decoding the current block according to the current intra prediction mode.