Two-dimensional palette coding for screen content coding

1. A method for video data coding with a palette mode, the method comprising:

generating a palette table predictor for a current coding unit based on a previous palette table associated with a previous coding unit and a previous palette table predictor associated with the previous coding unit, the palette table predictor comprising a plurality of predictor colors;

identifying a plurality of representative colors of the current coding unit;

comparing the plurality of representative colors of the current coding unit to the plurality of predictor colors, and for each representative color that is not in the palette table predictor:

identifying a corresponding predictor color in the palette table predictor,

selecting a palette color to include in a palette table associated with the current coding unit between the representative color and the corresponding predictor color based on their respective rate distortion costs,

determining whether to include the selected palette color in the palette table associated with the current coding unit, and

updating the palette table associated with the current coding unit based on the determining; and

palette coding the current coding unit using the palette table associated with the current coding unit.

2. The method of claim 1 , wherein selecting the palette color further comprises:

calculating a first rate distortion cost associated with including the representative color in the palette table associated with the current coding unit;

calculating a second rate distortion cost associated with including the corresponding predictor color in the palette table associated with the current coding unit; and

selecting the corresponding predictor color in the palette table predictor on a condition that the second rate distortion cost is less than the first rate distortion cost.

3. The method of claim 1 , wherein one or more colors in the palette table associated with the current coding unit are copied from one or more colors in the palette table predictor for the current coding unit.

4. The method of claim 1 , wherein at least one of the plurality of representative colors of the current coding unit is selected as an average of one or more colors associated with one or more pixels in the current coding unit.

5. The method of claim 4 , wherein the average is a weighted average, and wherein weights correspond to respective frequencies of occurrence of the one or more colors.

6. A video coder for coding a video data with a palette mode, the video coder comprising:

a processor configured to execute processor-executable instructions; and

a memory storing instructions that, when executed by the processor, cause the processor to:

generate a palette table predictor for a current coding unit based on a previous palette table associated with a previous coding unit and a previous palette table predictor associated with the previous coding unit, the palette table predictor comprising a plurality of predictor colors;

identify a plurality of representative colors of the current coding unit;

compare the plurality of representative colors of the current coding unit to the plurality of predictor colors, and for each representative color that is not in the palette table predictor:

identify a corresponding predictor color in the palette table predictor,

select a palette color to include in a palette table associated with the current coding unit between the representative color and the corresponding predictor color based on their respective rate distortion costs,

determine whether to include the selected palette color in the palette table associated with the current coding unit, and

update the palette table associated with the current coding unit based on the determining; and

palette code the current coding unit using the palette table associated with the current coding unit.

7. The video coder of claim 6 , wherein the memory stores further instructions that, when executed by the processor, cause the processor to:

calculate a first rate distortion cost associated with including the representative color in the palette table associated with the current coding unit; and

calculate a second rate distortion cost associated with including the corresponding predictor color in the palette table associated with the current coding unit, wherein the corresponding predictor color in the palette table predictor is selected to be included in the palette table associated with the current coding unit on a condition that the second rate distortion cost is less than the first rate distortion cost, and the representative color is selected to be included in the palette table associated with the current coding unit on a condition that the second rate distortion cost is greater than the first rate distortion cost.

8. The video coder of claim 6 , wherein one or more colors in the palette table associated with the current coding unit are copied from one or more colors in the palette table predictor for the current coding unit.

9. The video coder of claim 6 , wherein at least one of the plurality of representative colors of the current coding unit is selected as an average of one or more colors associated with one or more pixels in the current coding unit.

10. The video coder of claim 9 , wherein the average is a weighted average, and wherein weights correspond to respective frequencies of occurrence of the one or more colors.

11. The method of claim 1 , wherein generating the palette table predictor for the current coding unit further comprising:

merging the previous palette table associated with the previous coding unit and the previous palette table predictor associated with the previous coding unit.

12. The method of claim 11 , wherein the merging is limited by a maximum palette predictor size.

13. The method of claim 1 , wherein generating the palette table predictor for the current coding unit further comprising:

adding, to the palette table predictor for the current coding unit, a plurality of entries from the previous palette table associated with the previous coding unit.

14. The method of claim 13 , wherein generating the palette table predictor for the current coding unit further comprising:

adding, to the palette table predictor for the current coding unit, at least one entry from the previous palette table predictor associated with the previous coding unit after the plurality of entries from the previous palette table associated with the previous coding unit.

15. The method of claim 14 , wherein the at least one entry from the previous palette table predictor associated with the previous coding unit corresponds to a color that is not represented in the previous palette table associated with the previous coding unit.

16. The method of claim 1 , further comprising:

creating a palette index map that maps one or more pixels of the current coding unit to a color index in the palette table associated with the current coding unit; and

generating palette index map prediction data that indicates values in the palette index map associated with at least some portions of the current coding unit.

17. The video coder of claim 6 , wherein generating the palette table predictor for the current coding unit further comprising:

merging the previous palette table associated with the previous coding unit and the previous palette table predictor associated with the previous coding unit.

18. The video coder of claim 17 , wherein the merging is limited by a maximum palette predictor size.

19. The method of claim 2 , wherein the first rate distortion cost associated with including the representative color in the palette table is calculated as a function of distortion associated with coding a plurality of pixels in a corresponding color cluster with the representative color and a number of bits associated with coding the representative color.

20. The method of claim 2 , wherein the second rate distortion cost associated with including the predictor color in the palette table is calculated as a function of distortion associated with coding a plurality of pixels in a corresponding color cluster with the predictor color and a number of bits associated with including the predictor color in the palette table.

21. The method of claim 20 , wherein the number of bits associated with including the predictor color in the palette table is calculated based on a size of the palette table predictor and a size of the palette table.

22. The method of claim 2 , wherein determining whether to include the selected palette color in the palette table further comprises:

calculating a third rate distortion cost associated with excluding the selected palette color from the palette table associated with the current coding unit;

determining to include the selected palette color in the palette table associated with the current coding unit on a condition that the third rate distortion cost is greater than the first rate distortion cost or the second rate distortion cost; and

determining to exclude the selected palette color from the palette table associated with the current coding unit on a condition that the third rate distortion cost is less than the first rate distortion cost and less than the second rate distortion cost.

23. The method of claim 22 , wherein the third rate distortion cost associated with excluding the selected palette color from the palette table associated with the current coding unit is calculated as a function of a number of coding bits associated with coding the selected palette color as an escape color and a distortion associated with coding the selected palette color as an escape color.

24. The video coder of claim 7 , wherein the first rate distortion cost associated with including the representative color in the palette table is calculated as a function of distortion associated with coding a plurality of pixels in a corresponding color cluster with the representative color and a number of bits associated with coding the representative color.

25. The video coder of claim 7 , wherein the second rate distortion cost associated with including the predictor color in the palette table is calculated as a function of distortion associated with coding a plurality of pixels in a corresponding color cluster with the predictor color and a number of bits associated with including the predictor color in the palette table.

26. The video coder of claim 25 , wherein the number of bits associated with including the predictor color in the palette table is calculated based on a size of the palette table predictor and a size of the palette table.

27. The video coder of claim 7 , wherein the memory stores further instructions that, when executed by the processor, cause the processor to:

calculate a third rate distortion cost associated with excluding the selected palette color from the palette table associated with the current coding unit;

determine to include the selected palette color in the palette table associated with the current coding unit on a condition that the third rate distortion cost is greater than the first rate distortion cost or the second rate distortion cost; and

determine to exclude the selected palette color from the palette table associated with the current coding unit on a condition that the third rate distortion cost is less than the first rate distortion cost and less than the second rate distortion cost.

28. The video coder of claim 27 , wherein the third rate distortion cost associated with excluding the selected palette color from the palette table associated with the current coding unit is calculated as a function of a number of coding bits associated with coding the selected palette color as an escape color and a distortion associated with coding the selected palette color as an escape color.