Query rewrites for data-intensive applications in presence of run-time errors
An aspect of error handling includes a parsing block for pre-processing a document indexing application, a filtering block for discarding irrelevant documents, a transformation block to clean up and annotate input data by identifying a document language, and a processor configured for grouping inputs to collect documents for a same entity in a single spot. The processor processes a query operation. An aspect of error handling also includes a data package including a data result component that includes data generated based on successful completion of at least a portion of the query operation. The data package also includes an error information data component based on one or more errors encountered during processing of the query operation. An indicator separates the error information data from the data result. The system also includes a memory associated with a distributed file system for storing a final write output relating to the query operation.
1. A system comprising:
a parsing block for pre-processing of a document indexing application;
a filtering block for discarding irrelevant documents;
a transformation block to clean up and annotate input data by identifying at least one document language;
a processor configured for grouping inputs to collect documents for a same entity in a single spot, the processor processing a query operation;
a data package including a data result component, the data result component including data generated based on successful completion of at least a portion of the query operation; wherein the data package also includes an error information data component based on one or more errors encountered during processing of the query operation;
the processor continuing, as a function of at least one of a number of errors encountered and a class of error encountered, the query operation on data results relating to portions of the query operation that successfully completed;
an indicator to separate and distinguish the error information data from the data result;
a memory associated with a distributed file system for storing a final write output relating to the query operation; and
a compiler that uses the data package for generating a program to optimize processing time by establishing a declarative specification of error tolerance based on previous results.
2. The system of claim 1 , the query operation is performed using declarative reasoning.
3. The system of claim 2 , wherein the data result includes information and data generated before and after encountering the error and in performance of the query operation; the data result omitting data deemed affected due to encountering the error.
4. The system of claim 3 , wherein the error is a nondeterministic error.
5. The system of claim 1 , wherein the query operation is performed responsive to receiving a command request.
6. The system of claim 5 , wherein a plurality of query operations are performed responsive to one or more command requests.
7. The system of claim 6 , wherein the query operations are performed in parallel to one another.
8. The system of claim 7 , wherein individual data packages are generated for each query operation including separate data result and error information based upon encountering one or more errors.