Palette size constraint in palette mode for video compression system

1. A method in a video encoder comprising:

determining a sample amount of a current palette;

determining a number of previous elements from a palette predictor list to be reused in the current palette based on a binary vector;

determining a difference between the sample amount and the number of the previous elements to be reused;

setting the difference as a maximum number of new elements to be used in the current palette;

determining a set of new elements corresponding to the maximum number of new elements; and

transmitting the set of new elements.

2. The method of claim 1 , wherein:

the sample amount of the current palette includes a size of a coding unit for the current palette and a size of a prediction unit for the current palette, and

the maximum number of new elements is less than or equal to the sample amount of the current palette.

3. The method of claim 1 , further comprising, prior to determining the set of new elements:

determining a total number of escape pixels and reconstructed samples to be used in the current palette;

reducing the maximum number of new elements by the total number; and

setting the reduced maximum number as the maximum number of new elements.

4. The method of claim 3 , wherein the set of new elements is different from:

the previous elements, and

escape pixels and reconstructed samples associated with the total number.

5. The method of claim 3 , further comprising:

setting an encoding parameter for encoding the new elements such that a number of the encoded new elements is equal to the maximum number of new elements.

6. A computer-readable storage medium storing computer-readable instructions executable by one or more processors of a video encoder, that when executed by the one or more processors, cause the one or more processors to perform operations comprising:

determining a number of previous elements from a palette predictor list to be reused in a current palette based on a binary vector;

determining a sample amount of the current palette;

determining a difference between the sample amount and the number of the previous elements to be reused;

setting the difference as a maximum number of new elements to be used in the current palette;

determining a set of new elements corresponding to the maximum number of new elements; and

transmitting the set of new elements.

7. The computer-readable storage medium of claim 6 , wherein:

the sample amount of the current palette includes a size of a coding unit for the current palette and a size of a prediction unit for the current palette, and

the maximum number of new elements is less than or equal to the sample amount of the current palette.

8. The computer-readable storage medium of claim 6 , wherein the operations further comprise, prior to determining the set of new elements:

determining a total number of escape pixels and reconstructed samples to be used in the current palette;

reducing the maximum number of new elements by the total number; and

setting the reduced maximum number as the maximum number of new elements.

9. The computer-readable storage medium of claim 8 , wherein the set of new elements is different from:

the previous elements, and

escape pixels and reconstructed samples associated with the total number.

10. The computer-readable storage medium of claim 8 , wherein the operations further comprise:

setting an encoding parameter for encoding the new elements such that a number of the encoded new elements is equal to the maximum number of new elements.

11. A system for a video encoder comprising:

one or more processors; and

memory communicatively coupled to the one or more processors, the memory storing computer-executable modules executable by the one or more processors that, when executed by the one or more processors, perform associated operations, the computer-executable modules including:

a determination module configured to determine:

a number of previous elements from a palette predictor list to be reused in a current palette based on a binary vector,

a sample amount of a current palette,

a maximum number of new elements to be used in the current palette based on the sample amount by:

determining a difference between the sample amount and the number of the previous elements to be reused, and

setting the difference as the maximum number of new elements, and

a set of new elements corresponding to the maximum number of new elements; and

a transmitting module configured to transmit the set of new elements.

12. The system of claim 11 , wherein:

the sample amount of the current palette includes a size of a coding unit for the current palette and a size of a prediction unit for the current palette, and

the maximum number of new elements is less than or equal to the sample amount of the current palette.

13. The system of claim 11 , wherein the determination module is further configured to, prior to determining the set of new elements:

determine a total number of escape pixels and reconstructed samples to be used in the current palette;

reduce the maximum number of new elements by the total number; and

set the reduced maximum number as the maximum number of new elements.

14. The system of claim 13 , wherein the set of new elements is different from:

the previous elements, and

escape pixels and reconstructed samples associated with the total number.

15. The system of claim 13 , wherein the computer-executable modules further include:

a parameter module configured to set an encoding parameter for encoding the new elements such that a number of the encoded new elements is equal to the maximum number of new elements.