Executing structured queries on unstructured data
Technologies are described herein for executing queries expressed with reference to a structured query language against unstructured data. A user issues a structured query through a traditional structured data management (“SDM”) application. Upon receiving the structured query, an SDM driver analyzes the structured query and extracts a data structure from the unstructured data, if necessary. The structured query is then converted to an unstructured query based on the extracted data structure. The converted unstructured query may then be executed against the unstructured data. Results from the query are reorganized into structured data utilizing the extracted data structure and are then presented to the user through the SDM application.
1. A computer-implemented method, comprising:
transmitting a pilot query to an unstructured data system, the pilot query requesting that the unstructured data system identify a first set of one or more fields that the unstructured data system can use to search raw machine data stored in textual form, wherein a field is defined by an extraction rule specifying where to find a subportion of text within a larger segment of text of the raw machine data;
receiving the identified first set of one or more fields;
receiving, at a query converter, a structured query whose portions all correspond to a structured query language;
converting, by the query converter, the structured query into an unstructured query in an unstructured query language associated with searching the raw machine data in the unstructured data system, the unstructured query referencing at least one field in the first set of one or more fields; and
transmitting an instruction to the unstructured data system requesting that the unstructured data system execute the unstructured query against the raw machine data in the textual form;
wherein the method is performed by one or more computing devices.
2. The computer-implemented method of claim 1 , further comprising caching the identified first set of one or more fields.
3. The computer-implemented method of claim 1 , wherein requesting that the unstructured data system identify the first set of one or more fields that the unstructured data system can use to search the raw machine data causes the unstructured data system to identify the first set of one or more fields from a subset of the raw machine data that the unstructured data system can search.
4. The computer-implemented method of claim 1 , wherein requesting that the unstructured data system identify the first set of one or more fields that the unstructured data system can use to search the raw machine data causes the unstructured data system to identify the first set of one or more fields from a subset of the raw machine data that the unstructured data system can search, and wherein the subset of the raw machine data is of a user definable size.
5. The computer-implemented method of claim 1 , wherein the structured query comprises a Structured Query Language (“SQL”) query.
6. A computer-implemented system, comprising:
a processor coupled to tangible memory, the tangible memory including instructions that, when executed, cause the system to carry out actions including:
transmitting a pilot query to an unstructured data system, the pilot query requesting that the unstructured data system identify a first set of one or more fields that the unstructured data system can use to search raw machine data stored in textual form, wherein a field is defined by an extraction rule specifying where to find a subportion of text within a larger segment of text of the raw machine data;
receiving the identified first set of one or more fields;
receiving, at a query converter, a structured query whose portions all correspond to a structured query language;
converting, by the query converter, the structured query into an unstructured query in an unstructured query language associated with searching the raw machine data in the unstructured data system, the unstructured query referencing at least one field in the first set of one or more fields; and
transmitting an instruction to the unstructured data system requesting that the unstructured data system execute the unstructured query against the raw machine data in the textual form.
7. The computer-implemented system of claim 6 , further comprising caching the identified first set of one or more fields.
8. The computer-implemented system of claim 6 , wherein requesting that the unstructured data system identify the first set of one or more fields that the unstructured data system can use to search the raw machine data causes the unstructured data system to identify the first set of one or more fields from a subset of the raw machine data that the unstructured data system can search.
9. The computer-implemented system of claim 6 , wherein requesting that the unstructured data system identify the first set of one or more fields that the unstructured data system can use to search the raw machine data causes the unstructured data system to identify the first set of one or more fields from a subset of the raw machine data that the unstructured data system can search, and wherein the subset of the raw machine data is of a user definable size.
10. The computer-implemented system of claim 6 , wherein the structured query comprises a Structured Query Language (“SQL”) query.
11. A tangible computer-readable memory, comprising:
the tangible computer-readable memory including instructions that, when executed, cause a computer-implemented system to carry out actions including:
transmitting a pilot query to an unstructured data system, the pilot query requesting that the unstructured data system identify a first set of one or more fields that the unstructured data system can use to search raw machine data stored in textual form, wherein a field is defined by an extraction rule specifying where to find a subportion of text within a larger segment of text of the raw machine data;
receiving the identified first set of one or more fields;
receiving, at a query converter, a structured query whose portions all correspond to a structured query language;
converting, by the query converter, the structured query into an unstructured query in an unstructured query language associated with searching the raw machine data in the unstructured data system, the unstructured query referencing at least one field in the first set of one or more fields; and
transmitting an instruction to the unstructured data system requesting that the unstructured data system execute the unstructured query against the raw machine data in the textual form;
wherein the actions are performed by one or more computing devices.
12. The tangible computer-readable memory of claim 11 , further comprising instructions causing the computer-implemented system to carry out caching the identified first set of one or more fields.
13. The computer-implemented method of claim 11 , wherein requesting that the unstructured data system identify the first set of one or more fields that the unstructured data system can use to search the raw machine data causes the unstructured data system to identify the first set of one or more fields from a subset of the raw machine data that the unstructured data system can search.
14. The computer-implemented method of claim 11 , wherein requesting that the unstructured data system identify the first set of one or more fields that the unstructured data system can use to search the raw machine data causes the unstructured data system to identify the first set of one or more fields from a subset of the raw machine data that the unstructured data system can search, and wherein the subset of the raw machine data is of a user definable size.
15. The computer-implemented method of claim 11 , wherein the structured query comprises a Structured Query Language (“SQL”) query.