Network interface for distributed intelligence database system
View Patent ↗A database appliance in which two or more internetworked data storage units are used to coordinate the storage and retrieval of database records. One or more central database processing units are also associated with the data storage units. A network infrastructure provides the ability for the central database processors and storage processors to communicate as network nodes, with the network infrastructure using a communication protocol. A software application for executing database operations executes in a distributed fashion with portions of the database application executing on at least one central database processor and other portions executing on the data storage processors. At least a portion of the database application is implemented within and/or coordinated by a communication process that is executing the communication protocol. This coordination takes place such that data blocks are passed between the communication process and at least one portion of the database application process by passing data block reference information. In accordance with other aspects of the present invention, the communication process may have at least portions of the database application process executing within it. These database application operations executing within the same context as the communication process may include database operations such as join, sort, aggregate, restrict, reject, expression evaluation, statistical analysis or other operations.
1. A distributed data processing system for executing database operations comprising;
(a) two or more data storage processors for coordinating and processing the storage and retrieval of database records;
(b) at least one central database processor for processing database operations;
(c) a network infrastructure providing for network connection of the data storage processors and at least one central database processor as network nodes to support communication of data among network nodes, the network infrastructure using a communication protocol;
(d) a network interface configured to operate a communication process responsible for communicating data blocks in one or more messages communicated among network nodes, the communication process formatting the data blocks with a communication header;
(e) an application process for executing database operations including coordinating storage and retrieval of data blocks, with portions of the application process executing on the data storage processors, and portions of the application process executing on at least one central database processor;
(f) a memory coupled to the network interface and one of the network node processors, the application process and network interface having access to the memory such that the application process shares data blocks with the communication process by reference; and
(g) the application process storing database operation information within the communication header, and generating and passing data block pointer information to the communication process for referencing a given data block.
2. The system as in claim 1 wherein a data storage processor also serves as a central database processor.
3. The system as in claim 1 wherein a portion of the application process executes in the network interface.
4. The system as in claim l wherein the communication process performs database operations on received data packets prior to forwarding them to the application process.
5. The system as in claim 1 wherein the database operations include one or more from the group of join, sort, aggregate, restrict, reject, expression evaluation or statistical analysis.
6. The system as in claim 1 wherein the communication process performs checksum operations on received data packets prior to forwarding them to the application process.
7. The system as in claim 1 wherein the communication process in a receiver processes data packets to determine routing information prior to passing them to the application process.
8. The system as in claim 1 wherein the application process executes in a central processing unit in the central database processor.
9. The system as in claim 1 wherein the network interface includes multiple network interface components that are individually addressable via the network infrastructure and are associated with the central database processor to provide a parallel processing architecture for processing parts of database queries in parallel among the two or more data storage processors.
10. The system as in claim 1 wherein source data and database primitive operation results from the communication process are stored in the memory accessible to a network interface component executing the communication process as well as to an application process executing on a central database processor.
11. A device for processing database operations comprising:
(a) a processor for coordinating and processing the storage and retrieval of database records;
(b) a network interface configured to operate a communication process responsible for communicating data blocks in one or more messages communicated among network nodes, the communication process formatting the data blocks with a communication header;
(c) an application process for executing database operations including coordinating storage and retrieval of data blocks, at least a portion of the application process executing on the processor, wherein other portions of the application process executing on processors associated with other correspondin network nodes;
(d) a memory coupled to the network interface and the processor, the application process and network interface having access to the memory such that the application process can share data blocks with the communication process by reference; and
(e) the application process storing database operation information within the communication header, and generating and passing data block pointer information to the communication process for referencing a given data block.
12. The device as in claim 11 wherein the network interface executes a portion of the application process.
13. The device as in claim 11 wherein the communication process performs database operations on received data packets prior to forwarding them to the application process.
14. The device as in claim 13 wherein the database operations are further selected from a group including join, sort, aggregate, restrict, reject, expression evaluation, and statistical analysis.
15. The device as in claim 11 wherein the communication process processes performs checksum operations on received data packets prior to forwarding them to the application process.
16. The device as in claim 11 wherein the communication process processes data packets to determine routing information prior to passing them to the application process.
17. A method of executing database operations comprising,
operating, at a network interface, a communication process to communicate data blocks in one or more messages communicated among two or more data storage processors and at least one central database processor as network nodes, the communication process formatting the data blocks with a communication header;
operating an application process to execute database operations including coordinating storage and retrieval of data blocks, with portions of the application process executing on the data storage processors, and portions of the application process executing on the at least one central database processor, the application process storing database operation information within the communication header, and generating and passing data block pointer information to the communication process for referencing a given data block; and
storing data to a memory coupled to the network interface and one of the data storage processors, the application process and network interface having access to the memory such that the application process can share data blocks with, the communication process by reference to the pointer information.
18. The method as in claim 17 wherein the communication process performs database operations on received data packets prior to forwarding them to an application process executing on the receiver.
19. The method as in claim 18 wherein the database operations include one or more from the group of join, sort, aggregate, restrict, reject, expression evaluation or statistical analysis.
20. A computer program product for executing database operations comprising:
a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code configured to:
operate, at a network interface, a communication process to communicate data blocks in one or more messages communicated among two or more data storage processors and at least one central database processor as network nodes, the communication process formatting the data blocks with a communication header;
operate an application process to execute database operations including coordinating storage and retrieval of data blocks, with portions of the application process executing on the data storage processors, and portions of the application process executing on the at least one central database processor, the application process storing database operation information within the communication header, and generating and passing data block pointer information to the communication process for referencing a given data block; and
store data to a memory coupled to the network interface and one of the data storage processors, the application process and network interface having access to the memory such that the application process shares data blocks with the communication process by reference to the pointer information.