System and method for information slingshot over a network tapestry and granularity of a tick
Systems and methods for file transfer and processing in a network environment are disclosed. In one embodiment, the system may comprise one or more processors. The one or more processors may be coupled to a first device. The one or more processors may be configured to retrieve a file from a file queue. The file may be stored in a local store of the first device. The file may be transferred from a second remote device via Remote Direct Memory Access. The one or more processors may further be configured to determine if the file is complete. The one or more processors may further be configured to remove the file from the file queue, if the file is determined to be complete.
1. A system for file transfer and processing in a network environment, comprising:
one or more processors, coupled to a first device, configured to:
determine a distance between the first device and a second device located in a remote region;
in response to determining that the distance is greater than a predetermined threshold:
retrieve a first file from a first file queue, wherein the first file is stored in a first local store of the first device, wherein the first file is transferred from the second device via a Remote Direct Memory Access write process initiated by the second device, and wherein the first file comprises a request from the second device;
process the retrieved first file;
in response to processing the retrieved first file, transfer a second file to a second file queue stored in a second local store of the second device, the second local store being located in the remote region, wherein the second file is transferred to the second file queue via a Remote Direct Memory Access write process initiated by the first device, and wherein the second file comprises a response to the request.
2. The system of claim 1 , wherein the second remote device is a backbone exchange server.
3. The system of claim 1 , further comprising a dedicated link between the first device and the second remote device.
4. The system of claim 1 , wherein at least one file among the first file and the second file comprises a header, a body, and a footer.
5. The system of claim 4 , wherein the footer comprises a marker that indicates an end of the at least one file.
6. The system of claim 4 , wherein the at least one file comprises a key value pair.
7. The system of claim 4 , wherein the footer comprises checksum to validate integrity of the body.
8. The system of claim 1 , wherein the Remote Direct Memory Access is implemented based on at least one of Infiniband or fibre channel.
9. The system of claim 1 , wherein at least one file among the first file and the second file comprises executable code.
10. The system of claim 1 , wherein the one or more processors are further configured to: if the first file is determined to be complete, at least one of set a flag on the first file, move the first file to a folder or delete the first file from the first local store.