PROCESSING STRUCTURED AND UNSTRUCTURED DATA USING OFFLOAD PROCESSORS
Methods of processing structured data are disclosed that can include providing a plurality of XIMM modules connected to a memory bus in a first server, with the XIMM modules each respectively having a DMA slave module connected to the memory bus and an arbiter for scheduling tasks, with the XIMM modules further providing an in-memory database; and connecting a central processing unit (CPU) in the first server to the XIMM modules by the memory bus, with the CPU arranged to process and direct structured queries to the plurality of XIMM modules.
1 . A method of processing structured data, comprising the steps of:
providing a plurality of XIMM modules connected to a memory bus in a first server, with the XIMM modules each respectively having a DMA slave module connected to the memory bus and an arbiter for scheduling tasks, with the XIMM modules further providing an in-memory database; and
connecting a central processing unit (CPU) in the first server to the XIMM modules by the memory bus, with the CPU arranged to process and direct structured queries to the plurality of XIMM modules.
2 . The method of processing structured data of claim 1 , wherein the XIMM modules communicate with each other without requiring access to a processor of the first server.
3 . The method of processing structured data of claim 1 , wherein the XIMM modules are mounted on different servers in the same rack, and further comprising a top of the rack switch to mediate communication therebetween.
4 . The method of processing structured data of claim 1 , wherein a XIMM driver executes a mmap routine to transfer a query from the CPU to the XIMM in the form of memory reads/writes.
5 . The method of processing structured data claim 1 , wherein the XIMM module is configured for insertion into a DIMM socket, and the XIMM module further comprises offload processors connected to memory and a computational FPGA.
6 . A method of processing unstructured data, comprising the steps of:
providing a plurality of XIMM modules connected to a memory bus, with the XIMM modules each respectively having a DMA slave module connected to the memory bus and an arbiter for scheduling tasks, with the XIMM modules providing an in-memory database; and
connecting a central processing unit (CPU) to the XIMM modules by the memory bus, with the CPU arranged to process computationally intensive tasks data processing tasks while directing parallel computation tasks to the plurality of XIMM modules.
7 . The method of processing unstructured data of claim 6 , wherein a Map/Reduce algorithm is processed, with results of a Map step being stored and available in main memory, and collected through DMA operations and parsed by the XIMM modules.
8 . The method of processing unstructured data of claim 6 , wherein the XIMM modules together define a massively parallel I/O mid-plane defined by the XIMMs.
9 . The method of processing unstructured data of claim 6 , wherein a Map/Reduce algorithm is processed by multiple servers supporting XIMM modules, and intermediate (key, value) pairs are exchanged across the multiple servers through intelligent virtual switching of the XIMM modules.
10 . The method of processing unstructured data of claim 6 , wherein the XIMM modules are configured for insertion into a DIMM socket, and the XIMM module further comprises offload processors connected to memory and a computational FPGA.