Real-time indexing of data for analytics

1. A computer-implemented method for processing data, comprising:

obtaining, in a buffer, records to be included in a data segment as the records are generated, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics;

after the buffer reaches a threshold size, creating the data segment comprising at least one of a forward index and an inverted index for a column in the records; and

enabling querying of the data segment and a set of offline data segments comprising older records of the one or more metrics and the at least one dimension, which comprises:

obtaining a latest time associated with available records in the set of offline data segments prior to performing a query;

performing the query with the set of offline data segments for the available records; and

performing the query with the data segment for records occurring after the latest time.

2. The computer-implemented method of claim 1 , further comprising:

prior to creating the data segment:

periodically creating a snapshot of the records in the buffer as the buffer continually receives records; and

enabling querying of at least one of a latest snapshot and earlier snapshots.

3. The computer-implemented method of claim 2 , wherein the snapshot is created according to a refresh frequency associated with querying of records in at least one of a latest snapshot and earlier snapshots.

4. The computer-implemented method of claim 1 , further comprising:

reporting an offset of the data segment after the data segment is created.

5. The computer-implemented method of claim 1 , further comprising:

periodically creating offline data segments from the older records independently of creation of the data segment.

6. The computer-implemented method of claim 1 , further comprising:

deleting the data segment after a pre-specified period has passed after creation of the data segment, wherein querying the data segment was enabled within the pre-specified period.

7. A system for processing data, comprising:

a first segment-creation apparatus configured to:

obtain, in a buffer, records to be included in a data segment as the records are generated, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics;

after the buffer reaches a threshold size, create the data segment comprising at least one of a forward index and an inverted index for a column in the records; and

a query-management apparatus configured to enable querying of the data segment and a set of offline data segments comprising older records of the one or more metrics and the at least one dimension, which comprises:

obtaining a latest time associated with available records in the set of offline data segments prior to performing a query;

performing the query with the set of offline data segments for the available records; and

performing the query with the data segment for records occurring after the latest time.

8. The system of claim 7 , wherein prior to creating the data segment, the first segment-creation apparatus is further configured to:

periodically create a snapshot of the records in the buffer as the buffer continually receives records; and

enable querying of at least one of a latest snapshot and earlier snapshots.

9. The system of claim 7 , wherein the first segment-creation apparatus is further configured to:

report an offset of the data segment after the data segment is created.

10. The system of claim 7 , further comprising:

a second segment-creation apparatus configured to periodically create the offline data segments from the older records independently of creation of the data segment in the first segment-creation apparatus.

11. The system of claim 7 , wherein the first segment-creation apparatus is further configured to:

delete the data segment after a pre-specified period has passed after creation of the data segment, wherein querying the data segment was enabled within the pre-specified period.

12. A computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for processing data, the method comprising:

obtaining, in a buffer, records to be included in a data segment as the records are generated, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics;

after the buffer reaches a threshold size, creating the data segment comprising at least one of a forward index and an inverted index for a column in the records; and

enabling querying of the data segment and a set of offline data segments comprising older records of the one or more metrics and the at least one dimension, which comprises:

obtaining a latest time associated with available records in the set of offline data segments prior to performing a query;

performing the query with the set of offline data segments for the available records; and

performing the query with the data segment for records occurring after the latest time.

13. The computer-implemented method of claim 12 , the method further comprising:

prior to creating the data segment:

periodically creating a snapshot of the records in the buffer as the buffer continually receives records; and

enabling querying of at least one of a latest snapshot and earlier snapshots.

14. The computer-implemented method of claim 12 , the method further comprising:

reporting an offset of the data segment after the data segment is created.

15. The computer-implemented method of claim 12 , the method further comprising:

periodically creating the offline data segments from the older records independently of creation of the data segment.

16. The computer-implemented method of claim 12 , the method further comprising:

deleting the data segment after a pre-specified period has passed after creation of the data segment, wherein querying the data segment was enabled within the pre-specified period.