Method and system for high performance integration, processing and searching of structured and unstructured data using coprocessors
View Patent ↗Disclosed herein is a method and system for integrating an enterprise's structured and unstructured data to provide users and enterprise applications with efficient and intelligent access to that data. Queries can be directed toward both an enterprise's structured and unstructured data using standardized database query formats such as SQL commands. A coprocessor can be used to hardware-accelerate data processing tasks (such as full-text searching) on unstructured data as necessary to handle a query. Furthermore, traditional relational database techniques can be used to access structured data stored by a relational database to determine which portions of the enterprise's unstructured data should be delivered to the coprocessor for hardware-accelerated data processing.
1. A method of performing queries on unstructured data, the method comprising:
receiving a query;
accessing structured data in response to the query to identify a subset of unstructured data that is to be analyzed against the query; and
performing a query-specified data analysis operation on the identified subset of unstructured data to thereby generate data for a response to the query;
wherein the accessing step is conducted by a processor; and
wherein the performing step is conducted by a coprocessor.
2. The method of claim 1 wherein the coprocessor comprises a reconfigurable logic device.
3. The method of claim 2 wherein the reconfigurable logic device has firmware deployed thereon, the firmware being configured to perform the query-specified data analysis operation on the subset of unstructured data.
4. The method of claim 3 wherein the data analysis operation comprises a full-text search operation.
5. The method of claim 1 wherein the structured data is stored in a relational database.
6. The method of claim 5 further comprising retrieving the identified subset of unstructured data prior to the performing step, wherein the retrieving step comprises retrieving the identified subset of unstructured data from the relational database.
7. The method of claim 1 wherein the structured data comprises a metadata index corresponding to unstructured data.
8. An apparatus for handling a query directed toward structured and unstructured data, the apparatus comprising:
a processor configured to (1) execute relational engine software and (2) execute coprocessor interface software, wherein the relational engine software is configured to (1) apply a portion of a query that is directed toward structured data to a relational database to thereby identify a subset of unstructured data, and (2) invoke the coprocessor interface software upon encountering a portion of the query that is directed toward unstructured data, and wherein the coprocessor interface software is configured to invoke a coprocessor to perform a query-specified data processing operation on the identified subset of unstructured data.
9. The apparatus of claim 8 wherein the coprocessor interface software is further configured to pass data to the coprocessor to thereby enable the coprocessor to perform the query-specified data processing operation on the identified subset of unstructured data.
10. The apparatus of claim 9 wherein the relational engine software comprises SQL relational engine software.