Management of data segments for analytics queries

1. A computer-implemented method for processing data, comprising:

obtaining a set of records, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics;

creating a data segment comprising at least one of a forward index and an inverted index for a column in the records;

storing the data segment in network-accessible storage;

assigning the data segment to a partition;

enabling querying of the data segment through a query node associated with the partition;

using a messaging service to provide a first notification to the query node of the data segment in the partition;

using the first notification to transfer the data segment from the network-accessible storage to the query node;

upon identifying the data segment as a replacement for a previous version of the data segment in the partition:

obtaining segment information for the data segment from the messaging service;

using the segment information to provide a second notification of the replacement to the query node; and

using the second notification to replace the previous version of the data segment with the data segment at the query node.

2. The computer-implemented method of claim 1 , further comprising:

disabling querying of the data segment through the query node upon detecting an expiration of the data segment.

3. The computer-implemented method of claim 1 , further comprising:

upon detecting a failure associated with an availability of the data segment for querying through the query node, managing the failure using at least one of a monitoring alert and an auto-recovery mechanism.

4. The computer-implemented method of claim 3 , wherein the failure is associated with at least one of:

obtaining the data segment;

storing the data segment;

communication with the messaging service; and

transfer of the data segment to the query node.

5. The computer-implemented method of claim 1 , wherein the previous version of the data segment is associated with at least one of data corruption and data loss.

6. The computer-implemented method of claim 1 , wherein the set of records in the data segment is obtained over a pre-specified period of time.

7. A system for processing data, comprising:

a data queue configured to provide a set of records, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics;

a segment-creation apparatus configured to create a data segment comprising at least one of a forward index and an inverted index for a column in the records;

a segment-management apparatus configured to:

store the data segment in network-accessible storage; and

assign the data segment to a partition; enable querying of the data segment using the forward index or the inverted index;

use a messaging service to provide a first notification to the query node of the data segment in the partition; and

upon identifying the data segment as a replacement for a previous version of the data segment in the partition:

obtain segment information for the data segment from the messaging service;

use the segment information to provide a second notification of the replacement to the query node; and

a query node associated with the partition, wherein the query node is configured to:

use the first notification to transfer the data segment from the network-accessible storage to the query node;

use the second notification to replace the previous version of the data segment with the data segment at the query node.

8. The system of claim 7 , wherein the segment-management apparatus is further configured to:

disable querying of the data segment through the query node upon detecting an expiration of the data segment.

9. The system of claim 7 , wherein the previous version of the data segment is associated with at least one of data corruption and data loss.

10. The system of claim 7 , further comprising:

a failure-management apparatus configured to:

detect a failure associated with an availability of the data segment for querying through the query node; and

manage the failure using at least one of a monitoring alert and an auto-recovery mechanism.

11. A computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for processing data, the method comprising:

obtaining a set of records, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics;

creating a data segment comprising at least one of a forward index and an inverted index for a column in the records;

storing the data segment in network-accessible storage;

assigning the data segment to a partition; enabling querying of the data segment through a query node associated with the partition;

using a messaging service to provide a first notification to the query node of the data segment in the partition;

using the first notification to transfer the data segment from the network-accessible storage to the query node;

upon identifying the data segment as a replacement for a previous version of the data segment in the partition:

obtaining segment information for the data segment from the messaging service;

using the segment information to provide a second notification of the replacement to the query node; and

using the second notification to replace the previous version of the data segment with the data segment at the query node.

12. The computer-readable storage medium of claim 11 , the method further comprising:

disabling querying of the data segment through the query node upon detecting an expiration of the data segment.

13. The computer-readable storage medium of claim 11 , the method further comprising:

upon detecting a failure associated with an availability of the data segment for querying through the query node, managing the failure using at least one of a monitoring alert and an auto-recovery mechanism.

14. The computer-readable storage medium of claim 13 , wherein the failure is associated with at least one of:

obtaining the data segment;

storing the data segment;

communication with the messaging service; and

transfer of the data segment to the query node.