SYSTEM AND METHOD FOR INFORMATION SLINGSHOT OVER A NETWORK TAPESTRY AND GRANULARITY OF A TICK
Systems and methods for file transfer and processing in a network environment are disclosed. In one embodiment, the system may comprise one or more processors. The one or more processors may be coupled to a first device. The one or more processors may be configured to retrieve a file from a file queue. The file may be stored in a local store of the first device. The file may be transferred from a second remote device via Remote Direct Memory Access. The one or more processors may further be configured to determine if the file is complete. The one or more processors may further be configured to remove the file from the file queue, if the file is determined to be complete.
1 . A method for transferring data from remote network locations, comprising:
determining, by one or more hardware processors, a time interval associated with batch processing files transferred from a remote network location to a local file system; and
periodically, at the time interval:
retrieving, by the one or more hardware processors, a batch of files from the local file system at the time interval; and
batch processing, by the one or more hardware processors, the plurality of files in parallel.