Intelligent data storage and processing using FPGA devices
A system is disclosed that comprises a field programmable gate array (FPGA), a network interface, and hardware description code, wherein the hardware description code is compilable into a plurality of bit configuration files for loading onto the FPGA, wherein each bit configuration file defines a pipelined processing operation for a hardware template. The FPGA comprises configurable hardware logic, and the FPGA can be accessible over a network via the network interface for commanding the FPGA to load a bit configuration file from among the bit configuration files onto the FPGA to thereby configure hardware logic on the FPGA to perform the pipelined processing operation defined by the loaded bit configuration file, and wherein the FPGA is configured to (1) receive streaming data and (2) process the streaming data through the configured hardware logic to perform the pipelined processing operation defined by the loaded bit configuration file on the streaming data.
1. A system comprising:
a field programmable gate array (FPGA), the FPGA comprising configurable hardware logic;
a network interface; and
hardware description code that is compilable into a plurality of bit configuration files for loading onto the FPGA, wherein each bit configuration file defines a pipelined processing operation for a hardware template;
wherein the FPGA is accessible over a network via the network interface for commanding the FPGA to load a bit configuration file from among the bit configuration files onto the FPGA to configure hardware logic on the FPGA to perform the pipelined processing operation defined by the loaded bit configuration file; and
wherein the FPGA is configured to (1) receive streaming data and (2) process the streaming data through the configured hardware logic to perform the pipelined processing operation defined by the loaded bit configuration file on the streaming data.
2. The system of claim 1 further comprising:
a processor; and
a bus that links the processor with the FPGA; and
wherein the FPGA is accessible over the network interface via the processor and the bus.
3. The system of claim 2 wherein the FPGA is configured to offload a plurality of processing tasks from the processor, and wherein the network interface, processor, FPGA, and bus are part of a computer server.
4. The system of claim 1 wherein the hardware description code is synthesizable into gate level description code, and wherein the gate level description code is convertible into the bit configuration files.
5. The system of claim 1 wherein the hardware description code includes hardware description code that is compilable into a bit configuration file that defines a compression pipeline hardware template.
6. The system of claim 1 wherein the hardware description code includes hardware description code that is compilable into a bit configuration file that defines a decompression pipeline hardware template.
7. The system of claim 1 wherein the hardware description code includes hardware description code that is compilable into a bit configuration file that defines an encryption pipeline hardware template.
8. The system of claim 1 wherein the hardware description code includes hardware description code that is compilable into a bit configuration file that defines a decryption pipeline hardware template.
9. The system of claim 1 wherein the hardware description code includes hardware description code that is compilable into a bit configuration file that defines a search pipeline hardware template.
10. The system of claim 1 wherein the hardware description code includes hardware description code that is compilable into a bit configuration file that defines an image processing pipeline hardware template, wherein the image processing pipeline hardware template comprises a matching engine that computes a similarity between streaming image data and an image key.
11. The system of claim 1 wherein the hardware description code includes hardware description code that is compilable into a bit configuration file that defines a data reduction hardware template.
12. The system of claim 1 wherein the hardware description code includes hardware description code that is compilable into a bit configuration file that defines a processing pipeline hardware template for determining a similarity between DNA sequences.
13. The system of claim 1 wherein the hardware description code includes hardware description code that is compilable into a plurality of bit configuration files that define a plurality of different hardware templates, wherein the different hardware templates comprise at least two members of the group consisting of (1) a compression pipeline hardware template, (2) a decompression pipeline hardware template, (3) an encryption pipeline hardware template, (4) a decryption pipeline hardware template, (5) a search pipeline hardware template, (6) an image processing pipeline hardware template, wherein the image processing pipeline hardware template comprises a matching engine that computes a similarity between streaming image data and an image key, (7) a data reduction hardware template, and (8) a processing pipeline hardware template for determining a similarity between DNA sequences.
14. The system of claim 1 wherein the FPGA is also accessible over the network interface for commanding the FPGA to (1) load a different bit configuration file from among the bit configuration files onto the FPGA to thereby differently configure hardware logic on the FPGA to perform the pipelined processing operation defined by the loaded different bit configuration file, (2) receive additional streaming data via the network interface, and (3) process the additional streaming data through the differently configured hardware logic to perform the pipelined processing operation defined by the loaded different bit configuration file on the additional streaming data.
15. The system of claim 1 further comprising a memory, the memory configured to store the hardware description code, and wherein the memory is remote from the FPGA via the network.
16. The system of claim 1 further comprising a memory, the memory configured to store the bit configuration files, wherein the memory is linked to the FPGA and co-resident on a board with the FPGA.
17. The system of claim 1 wherein the FPGA is further configured to receive the streaming data via the network interface.
18. The system of claim 1 further comprising (1) a storage medium from which the FPGA receives the streaming data and (2) a processor;
wherein the storage medium comprises a mass storage medium in form of a disk, the disk having a plurality of discontiguous arcs for data storage thereon, wherein a plurality of data files are stored in the mass storage medium, each data file being stored as a sequence of segments, each segment having a size that is a power of 2; and
wherein the processor is configured to (1) write the data files to the mass storage medium as the sequence of segments and (2) manage a flow of a plurality of the data files to the FPGA for processing as the streaming data.
19. A system comprising:
a field programmable gate array (FPGA), the FPGA comprising configurable hardware logic;
a network interface; and
gate level description code that is convertible into a plurality of bit configuration files for loading onto the FPGA, wherein each bit configuration file defines a pipelined processing operation for a hardware template;
wherein the FPGA is accessible over a network via the network interface for commanding the FPGA to load a bit configuration file from among the bit configuration files onto the FPGA to configure hardware logic on the FPGA to perform the pipelined processing operation defined by the loaded bit configuration file; and
wherein the FPGA is configured to (1) receive streaming data and (2) process the streaming data through the configured hardware logic to perform the pipelined processing operation defined by the loaded bit configuration file on the streaming data.
20. A system comprising:
a field programmable gate array (FPGA), the FPGA comprising configurable hardware logic;
a network interface; and
a plurality of bit configuration files for loading onto the FPGA, wherein each bit configuration file defines a pipelined processing operation for a hardware template;
wherein the FPGA is accessible over a network via the network interface for commanding the FPGA to load a bit configuration file from among the bit configuration files onto the FPGA to configure hardware logic on the FPGA to perform the pipelined processing operation defined by the loaded bit configuration file; and
wherein the FPGA is configured to (1) receive streaming data and (2) process the streaming data through the configured hardware logic to perform the pipelined processing operation defined by the loaded bit configuration file on the streaming data.