Fast serialization for data transfer

1. A computer-implemented method for improving data transferring efficiency from a first database to a second database, comprising:

analyzing a column of data stored in the first database to determine at least one serialization parameter, the at least one serialization parameter comprises a repetition counter or a replication counter;

determining a data serialization scheme associated with the data based on the at least one serialization parameter, wherein the data serialization scheme comprises at least a repetition scheme or a replication scheme, wherein the repetition scheme comprises:

writing the repetition counter to a transfer medium;

serializing at least one value in a value set to the transfer medium;

determining an index that corresponds to a data value; and

writing the index to the transfer medium;

determining a variable serialization scheme associated with the data based on a variable type of the data;

repeating the analyzing, the determining of the data serialization scheme, and the determining of the variable serialization scheme for each column of the data stored in the first database;

serializing the data stored in the first database using the data serialization scheme and the variable serialization scheme; and

transferring the serialized data to the second database.

2. The method of claim 1 , further comprising:

prior to the analyzing, determining whether the first database is a row-based database; and

if the first database is row-based, transferring the first database into a column-based database.

3. The method of claim 1 , wherein determining the data serialization scheme further comprises:

if the repetition counter is smaller than a predetermined limit, determining that the data serialization scheme comprises the repetition scheme; and

if the repetition counter is greater than or equal to the predetermined limit and the replication counter is larger than a predetermined threshold, determining that the data serialization scheme comprises the republication scheme.

4. The method of claim 1 , wherein determining the variable serialization scheme further comprises:

if the variable type is integer, determining that the variable serialization comprises an integer scheme; and

if the variable type is character, determining that the variable serialization comprises a character scheme.

5. The method of claim 1 , wherein the replication scheme further comprises:

setting an additional value counter to zero;

writing a data value to a transfer medium;

reading a next data value;

in response to a determination that the next data value is equal to the data value, increasing the additional value counter by one; and

writing the additional value counter to the transfer medium.

6. A system for improving data transferring efficiency from a first database to a second database, comprising:

a memory; and

at least one hardware processor interoperably coupled with the memory and configured to:

analyze a column of data stored in the first database to determine at least one serialization parameter, the at least one serialization parameter comprises a repetition counter or a replication counter;

determine a data serialization scheme associated with the data based on the at least one serialization parameter, wherein the data serialization scheme comprises at least a repetition scheme or a replication scheme, wherein the repetition scheme comprises:

writing the repetition counter to a transfer medium;

serializing at least one value in a value set to the transfer medium;

determining an index that corresponds to a data value; and

writing the index to the transfer medium;

determine a variable serialization scheme associated with the data based on a variable type of the data;

repeat the analyzing, the determining of the data serialization scheme, and the determining of the variable serialization scheme for each column of the data stored in the first database;

serialize the data stored in the first database using the data serialization scheme and the variable serialization scheme; and

transfer the serialized data to the second database.

7. The system of claim 6 , wherein the at least one hardware processor is further configured to:

prior to the analyzing, determine whether the first database is a row-based database; and

if the first database is row-based, transfer the first database into a column-based database.

8. The system of claim 6 , wherein determining the data serialization scheme further comprises:

if the repetition counter is smaller than a predetermined limit, determining that the data serialization scheme comprises the repetition scheme; and

if the repetition counter is greater than or equal to the predetermined limit and the replication counter is larger than a predetermined threshold, determining that the data serialization scheme comprises the republication scheme.

9. The system of claim 6 , wherein determining the variable serialization scheme further comprises:

if the variable type is integer, determining that the variable serialization comprises an integer scheme; and

if the variable type is character, determining that the variable serialization comprises a character scheme.

10. The system of claim 6 , wherein the replication scheme further comprises:

setting an additional value counter to zero;

writing a data value to a transfer medium;

reading a next data value;

in response to a determination that the next data value is equal to the data value, increasing the additional value counter by one; and

writing the additional value counter to the transfer medium.

11. A non-transitory, computer-readable medium storing computer-readable instructions for improving data transferring efficiency from a first database to a second database, the instructions executable by a computer and configured to:

analyze a column of data stored in the first database to determine at least one serialization parameter, the at least one serialization parameter comprises a repetition counter or a replication counter;

determine a data serialization scheme associated with the data based on the at least one serialization parameter, wherein the data serialization scheme comprises at least a repetition scheme or a replication scheme, wherein the repetition scheme comprises:

writing the repetition counter to a transfer medium;

serializing at least one value in a value set to the transfer medium;

determining an index that corresponds to a data value; and

writing the index to the transfer medium;

determine a variable serialization scheme associated with the data based on a variable type of the data;

repeat the analyzing, the determining of the data serialization scheme, and the determining of the variable serialization scheme for each column of the data stored in the first database;

serialize the data stored in the first database using the data serialization scheme and the variable serialization scheme; and

transfer the serialized data to the second database.

12. The non-transitory, computer-readable medium of claim 11 , wherein the instructions are further configured to:

prior to the analyzing, determine whether the first database is a row-based database; and

if the first database is row-based, transfer the first database into a column-based database.

13. The non-transitory, computer-readable medium of claim 11 , wherein determining the data serialization scheme further comprises:

if the repetition counter is smaller than a predetermined limit, determining that the data serialization scheme comprises the repetition scheme; and

if the repetition counter is greater than or equal to the predetermined limit and the replication counter is larger than a predetermined threshold, determining that the data serialization scheme comprises the republication scheme.

14. The non-transitory, computer-readable medium of claim 11 , wherein determining the variable serialization scheme further comprises:

if the variable type is integer, determining that the variable serialization comprises an integer scheme; and

if the variable type is character, determining that the variable serialization comprises a character scheme.

15. The non-transitory, computer-readable medium of claim 11 , wherein the repetition scheme comprises:

writing the repetition counter to a transfer medium;

serializing at least one value in a value set to the transfer medium;

determining an index that corresponds to a data value; and

writing the index to the transfer medium.