Method of palette predictor signaling for video coding

1. A method of coding a block of video data using palette coding for a video coding system, the method comprising:

receiving input data associated with a current coding unit having a current block size;

selecting a current coding mode based on the current block size and a pre-defined block size, wherein selecting the current coding mode comprises:

if the current block size is larger than the pre-defined block size, selecting a current coding mode from a first coding group excluding a palette coding mode by skipping a flag for the palette mode;

if the current block size is equal to or smaller than the pre-defined block size, selecting the current coding mode from a second coding group including the palette coding mode by signaling or parsing the flag for the palette mode; and

encoding or decoding the current coding unit according to the current coding mode selected.

2. The method of claim 1 , wherein the pre-defined block size corresponds to 32×32.

3. The method of claim 1 , wherein the pre-defined block size corresponds to maximum transform unit size.

4. The method of claim 3 , wherein a palette mode flag is inferred to be zero to indicate no palette coding mode being used for the current coding unit if the current block size is larger than the pre-defined block size.

5. The method of claim 3 , wherein a palette mode flag is restricted to be zero to indicate no palette coding mode being used for the current coding unit.

6. The method of claim 1 , wherein if a palette mode flag indicates that the palette coding mode being used and the current coding unit is larger than the pre-defined block size, the current coding unit is divided into multiple blocks that size equal to the pre-defined block size and each block is coded by the palette coding mode.

7. The method of claim 6 , wherein each block uses individual palette coding syntax.

8. The method of claim 1 further comprises:

when the current coding mode selected for encoding the current coding unit is the palette coding mode, determining a palette related size corresponding to one or a combination of a reused major color size, a new major color size and a total major color size, binarizing the palette related size to generate a binarized palette related size according to one Kth order Exp-Golomb code, one unary code plus one Kth order Exp-Golomb code, or one truncated unary code plus one Kth order Exp-Golomb code, and applying entropy coding to the binarized palette related size to generate a coded palette related size; and

when the current coding mode selected for decoding the current coding unit is the palette coding mode, recovering the palette related size by applying said one Kth order Exp-Golomb code, said one unary code plus one Kth order Exp-Golomb code, or said one truncated unary code plus one Kth order Exp-Golomb code to the binarized palette related size recovered from a bitstream including the current coding unit, and recovering a current palette table for the current coding unit based on a palette predictor and palette information including the palette related size;

wherein the reused major color size represents a first number of major colors in the current palette table that reuse major colors in the palette predictor, the new major color size represents a second number of major colors in the current palette table not predicted by the palette predictor, and the total major color size represents a third number of major colors present in the current coding unit.

9. The method of claim 1 further comprises:

when the current coding mode selected for encoding the current coding unit is the palette coding mode and if a current pixel of the current coding unit is coded in a copy mode, determining a palette run type for the current pixel and encoding the palette run type using context adaptive entropy coding with a single context without dependency on any previously encoded palette run type associated with any previous pixel, wherein the palette run type corresponds to either a copy-above mode or a copy-index mode; and

when the current coding mode selected for decoding the current coding unit is the palette coding mode, decoding the palette run type for the current pixel using the context adaptive entropy coding with the single context without dependency on any previously decoded palette run type associated with any previous pixel, reconstructing the current pixel and one or more following pixels according to the copy-above mode if the palette run type decoded is the copy-above mode, and reconstructing the current pixel and one or more following pixels according to the copy-index mode if the palette run type decoded is the copy-index mode.

10. The method of claim 1 further comprises:

when the current coding mode selected for encoding or decoding the current coding unit is the palette coding mode, determining a previous pixel for a current pixel within the current coding unit according to a scanning order;

determining a palette set for the current coding unit;

if the previous pixel corresponds to a copy-index mode, removing a redundant index corresponding to previous-pixel index from the palette set to form an updated palette set;

if the previous pixel corresponds to a copy-above mode, skipping redundant index removal and using the palette set as the updated palette set; and

encoding or decoding a current index of the current pixel based on the updated palette set.

11. The method of claim 1 , wherein the first group comprises an intra mode and excludes the palette mode, and wherein the second group comprises the intra mode and the palette mode.