Data beacon pulser(s) powered by information slingshot
Systems and methods for providing data beacons are disclosed. In some embodiments the system can include a first node and a second node. Each node includes a read queue, a write queue and a parallel file system. Data is written from the write queue on the first node to the parallel file system on the second node and from the write queue on the second node to the parallel file system on the first node. The read queue on each node receives data from the parallel file system on the node itself.
1. A method comprising:
fetching, by one or more hardware processors, from a first parallel file system, first data comprising information associated with a beacon pulse, wherein the beacon pulse is received at the first parallel file system from a write queue associated with a second parallel file system; and
placing, by the one or more hardware processors, the first data in a read queue.
2. The method of claim 1 , wherein the beacon pulse includes information from a source of changing information.
3. The method of claim 2 , wherein the changing information comprises one or more of market information, weather information, wind speed information, pollution measurement information, or content delivery network (CDN) replication information.
4. The method of claim 2 , wherein the beacon pulse comprises a snapshot of the changing information.
5. The method of claim 2 , wherein the beacon pulse comprises a differential that identifies one or more changes to the changing information over a predetermined interval.
6. The method of claim 5 , wherein the predetermined interval corresponds to an interval since a previous beacon pulse was transmitted.
7. The method of claim 1 , wherein a plurality of beacon pulses are received at the first parallel file system, and wherein data comprising information associated with the plurality of data beacons are fetched from the first parallel file system and placed in the read queue.
8. The method of claim 7 , wherein the plurality of beacon pulses are received at a set frequency.
9. The method of claim 8 , wherein the plurality of beacon pulses are received at variable intervals.
10. The method of claim 1 , wherein the beacon pulse is received over a long-distance network hop.
11. The method of claim 10 , wherein the first data in the first read queue is served to one or more clients in a target region via a communication protocol that comprises round trip communications with the one or more clients.
12. A system comprising:
a non-transitory memory; and
one or more hardware processors configured to read instructions from the non-transitory memory that, when executed, cause the one or more hardware processors to perform operations comprising:
fetching, from a first parallel file system, first data comprising information associated with a beacon pulse, wherein the beacon pulse is received at the first parallel file system from a write queue associated with a second parallel file system; and
placing the first data in a read queue.
13. The system of claim 12 , wherein the beacon pulse includes information from a source of changing information.
14. The system of claim 13 , wherein the changing information comprises one or more of market information, weather information, wind speed information, pollution measurement information, or content delivery network (CDN) replication information.
15. The system of claim 13 , wherein the beacon pulse comprises a differential that identifies one or more changes to the changing information over a predetermined interval.
16. The system of claim 12 , wherein a plurality of beacon pulses are received at the first parallel file system, and wherein data comprising information associated with the plurality of data beacons are fetched from the first parallel file system and placed in the read queue.
17. The system of claim 12 , wherein the beacon pulse is received over a long-distance network hop.
18. A non-transitory computer-readable medium storing instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to perform operations comprising:
fetching, from a first parallel file system, at predetermined intervals, first data comprising information associated with a plurality of beacon pulses, wherein the plurality of beacon pulses are received at the first parallel file system from a write queue associated with a second parallel file system; and
placing the first data in a read queue at the predetermined intervals.
19. The non-transitory computer-readable medium of claim 18 , wherein the plurality of beacon pulses are received at a set frequency.
20. The non-transitory computer-readable medium of claim 18 , wherein the beacon pulse is received over a long-distance network hop.