APPARATUSES, METHODS AND SYSTEMS FOR EFFICIENT AD-HOC QUERYING OF DISTRIBUTED DATA
The APPARATUSES, METHODS AND SYSTEMS FOR EFFICIENT AD-HOC QUERYING OF DISTRIBUTED DATA (“RTC”) provides a platform that, in various embodiments, is configurable to provide fast ad-hoc querying against large volumes of data. In one embodiment, the RTC is configurable to select a subset of fields from raw data in association with a domain and compact the corresponding data. Such packed records may be distributed to one or more worker nodes, which maintain the records and associated indexes. A master server facilitates query processing across the worker nodes.
1 . A processor-implemented method, comprising:
receiving a raw data record;
identifying at least one domain;
identifying a plurality of data fields associated with the at least one domain;
isolating domain data associated with the plurality of data fields from the raw data record; and
compacting the domain data to produce compacted data.
2 . The method of claim 1 , wherein the compacted data is configured in a binary format.
3 . The method claim 1 , further comprising:
providing the compacted data to at least one storage node in a distributed storage network, the at least one storage node being associated with the at least one domain.
4 . The method of claim 3 , further comprising:
receiving a query;
identifying at least one query domain associated with the query;
submitting a request for data associated with the query to the at least one storage node based on the at least one query domain.
5 . A system, comprising:
a processor;
a memory disposed in communication with the processor storing instructions causing the processor to:
receive a raw data record;
identify at least one domain;
identify a plurality of data fields associated with the at least one domain;
isolate domain data associated with the plurality of data fields from the raw data record; and
compact the domain data to produce compacted data.
6 . A processor-accessible non-transitory medium storing processor-issuable instructions, comprising:
receive a raw data record;
identify at least one domain;
identify a plurality of data fields associated with the at least one domain;
isolate domain data associated with the plurality of data fields from the raw data record; and
compact the domain data to produce compacted data.