Method and system for data indexing and reporting

1. A method implemented on a machine having at least one processor, storage, and a communication platform capable of connecting to a network for indexing user records, the method comprising:

obtaining a plurality of user records, wherein each user record is associated with one of a plurality of user identifiers and one or more of a plurality of data-segments, wherein the plurality of data-segments are variable length arrays;

for each of the plurality of data-segments,

determining a list of user identifiers associated with the data-segment, wherein the user identifiers in the list are sorted,

computing a relative distance for each user identifier in the sorted list excluding a first user identifier in the sorted list, wherein the relative distance is a value difference between the user identifier in the sorted list and a previous user identifier that immediately precedes the user identifier in the sorted list,

classifying, based on a magnitude of the relative distance, the relative distance of each user identifier into one of a plurality of different value ranges,

increasing the relative distance of each user identifier by a value that varies depending on the value range associated with the relative distance,

encoding the increased relative distance of each user identifier into a binary form, wherein a length of the binary form varies depending on the value range associated with the relative distance; and

generating, based on the encoded relative distances, an offset stream utilized to respond to a query from a user, wherein the offset stream includes offset values for the plurality of data-segments, wherein an offset value of a corresponding data-segment indicates an index of a bit stream where information of a first user identifier associated with the corresponding data-segment is stored, and wherein a report is generated as a response to the query and is presented to the user.

2. The method of claim 1 , further comprising determining a number of bits needed to store the encoded relative distance in the bit stream, wherein

if the relative distance is classified into a first value range, the number of bits used to store the encoded relative distance is a byte,

if the relative distance is classified into a second value range, the number of bits used to store the encoded relative distance is two bytes, and

if the relative distance is classified into a third value range, the number of bits used to store the encoded relative distance is three bytes.

3. The method of claim 1 , further comprising determining a number of bits needed to store the encoded relative distance in the bit stream,

wherein the step of determining the number of bits further comprises:

encoding the relative distance in accordance with a model, wherein an encoding parameter of the model is selected based on the relative distance.

4. The method of claim 3 , wherein a first two bits of the encoded relative distance indicates the number of bits used to store the encoded relative distance of the user identifier.

5. The method of claim 1 , further comprising:

obtaining the query from the user;

selecting one or more user identifiers from the bit stream that satisfy at least one rule associated with the query; and

generating the report to be presented to the user based on the selection.

6. The method of claim 1 , wherein information of the first user identifier associated with the data segment corresponds to the first user identifier being expressed in a binary format of length three bytes.

7. The method of claim 1 , wherein each of the user identifiers is a corresponding number.

8. A non-transitory computer readable medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method for indexing user records, the method comprising:

obtaining a plurality of user records, wherein each user record is associated with one of a plurality of user identifiers and one or more of a plurality of data-segments, wherein the plurality of data-segments are variable length arrays;

for each of the plurality of data-segments,

determining a list of user identifiers associated with the data-segment, wherein the user identifiers in the list are sorted,

computing a relative distance for each user identifier in the sorted list excluding a first user identifier in the sorted list, wherein the relative distance is a value difference between the user identifier in the sorted list and a previous user identifier that immediately precedes the user identifier in the sorted list,

classifying, based on a magnitude of the relative distance, the relative distance of each user identifier into one of a plurality of different value ranges,

increasing the relative distance of each user identifier by a value that varies depending on the value range associated with the relative distance,

encoding the increased relative distance of each user identifier into a binary form, wherein a length of the binary form varies depending on the value range associated with the relative distance; and

generating, based on the encoded relative distances, an offset stream utilized to respond to a query from a user, wherein the offset stream includes offset values for the plurality of data-segments, wherein an offset value of a corresponding data-segment indicates an index of a bit stream where information of a first user identifier associated with the corresponding data-segment is stored, and wherein a report is generated as a response to the query and is presented to the user.

9. The medium of claim 8 , wherein the method further comprises determining a number of bits needed to store the encoded relative distance in the bit stream, wherein

if the relative distance is classified into a first value range, the number of bits used to store the encoded relative distance is a byte,

if the relative distance is classified into a second value range, the number of bits used to store the encoded relative distance is two bytes, and

if the relative distance is classified into a third value range, the number of bits used to store the encoded relative distance is three bytes.

10. The medium of claim 8 , wherein the method further comprises determining a number of bits needed to store the encoded relative distance in the bit stream, and the step of determining the number of bits further comprises:

encoding the relative distance in accordance with a model, wherein an encoding parameter of the model is selected based on the relative distance.

11. The medium of claim 10 , wherein a first two bits of the encoded relative distance indicates the number of bits used to store the encoded relative distance of the user identifier.

12. The medium of claim 8 , wherein the method further comprises:

obtaining the query from the user;

selecting one or more user identifiers from the bit stream that satisfy at least one rule associated with the query; and

generating the report to be presented to the user based on the selection.

13. The medium of claim 8 , wherein information of the first user identifier associated with the data segment corresponds to the first user identifier being expressed in a binary format of length three bytes.

14. The medium of claim 8 , wherein each of the user identifiers is a corresponding number.

15. A system having at least one processor, storage, and a communication platform capable of connecting to a network for indexing user records, the system comprising:

a data receiving unit, implemented by the at least one processor, configured for obtaining a plurality of user records, wherein each user record is associated with one of a plurality of user identifiers and one or more of a plurality of data-segments, wherein the plurality of data- segments are variable length arrays;

a segment population unit, implemented by the at least one processor, configured for determining, for each of the plurality of data-segments, a list of user identifiers associated with the data-segment, wherein the user identifiers in the list are sorted,

a relative distance computing unit, implemented by the at least one processor, configured for computing, for each data-segment, a relative distance for each user identifier in the sorted list excluding a first user identifier in the sorted list, wherein the relative distance is a value difference between the user identifier in the sorted list and a previous user identifier that immediately precedes the user identifier in the sorted list;

a relative distance encoder, implemented by the at least one processor, configured for:

classifying, for each data-segment, based on a magnitude of the relative distance, the relative distance of each user identifier into one of a plurality of different value ranges,

increasing the relative distance of each user identifier by a value that varies depending on the value range associated with the relative distance, and

encoding the increased relative distance of each user identifier into a binary form, wherein a length of the binary form varies depending on the value range associated with the relative distance; and

a data stream generator, implemented by the at least one processor, configured for generating, based on the encoded relative distances, an offset stream utilized to respond to a query from a user, wherein the offset stream includes offset values for the plurality of data-segments, wherein an offset value of a corresponding data-segment indicates an index of a bit stream where information of a first user identifier associated with the corresponding data-segment is stored, and wherein a report is generated as a response to the query and is presented to the user.

16. The system of claim 15 , wherein the relative distance encoder is further configured for determining a number of bits needed to store the encoded relative distance in the bit stream, and wherein

if the relative distance is classified into a first value range, the number of bits used to store the encoded relative distance is a byte,

if the relative distance is classified into a second value range, the number of bits used to store the encoded relative distance is two bytes, and

if the relative distance is classified into a third value range, the number of bits used to store the encoded relative distance is three bytes.

17. The system of claim 15 , wherein the relative distance encoder is further configured for:

encoding the relative distance in accordance with a model, wherein an encoding parameter of the model is selected based on the relative distance.

18. The system of claim 17 , wherein a first two bits of the encoded relative distance indicates the number of bits used to store the encoded relative distance of the user identifier.

19. The system of claim 15 , further comprising a query processing unit, implemented by the at least one processor, configured for:

obtaining the query from the user;

selecting one or more user identifiers from the bit stream that satisfy at least one rule associated with the query; and

generating the report to be presented to the user based on the selection.

20. The system of claim 15 , wherein each of the user identifiers is a corresponding number.