Data beacon pulser(s) powered by information slingshot
Systems and methods for providing data beacons are disclosed. In some embodiments the system can include a first node and a second node. Each node includes a read queue, a write queue and a parallel file system. Data is written from the write queue on the first node to the parallel file system on the second node and from the write queue on the second node to the parallel file system on the first node. The read queue on each node receives data from the parallel file system on the node itself.
1. A method comprising:
receiving, by one or more processors, first data from a first server;
placing, by the one or more processors, the first data in a write queue associated with a first parallel file system; and
sending, by one or more hardware processors, the first data from the write queue to a second parallel file system via a beacon pulse.
2. The method of claim 1 , wherein the first data comprises information from a source of changing information.
3. The method of claim 2 , wherein the changing information comprises one or more of market information, weather information, wind speed information, pollution measurement information, or content delivery network (CDN) replication information.
4. The method of claim 2 , wherein the beacon pulse comprises a snapshot of the changing information.
5. The method of claim 2 , wherein the beacon pulse comprises a differential that identifies one or more changes to the changing information over a predetermined interval.
6. The method of claim 5 , wherein the predetermined interval corresponds to an interval since a previous beacon pulse was transmitted.
7. The method of claim 1 , wherein the first parallel file system is located in a first region, and the second parallel file system is located in a second region.
8. The method of claim 7 , further comprising sending, by the one or more processors, the first data to a third parallel file system located in a third region via the beacon pulse.
9. The method of claim 1 , further comprising sending, by the one or more processors, additional data in the write queue to the second parallel file system via a plurality of beacon pulses.
10. The method of claim 9 , wherein the plurality of beacon pulses are transmitted at a set frequency.
11. The method of claim 9 , wherein the plurality of beacon pulses are transmitted at variable intervals.
12. A system comprising:
a non-transitory memory; and
one or more hardware processors configured to read instructions from the non-transitory memory that, when executed, cause the one or more hardware processors to perform operations comprising:
receiving first data from a first server;
placing the first data in a write queue associated with a first parallel file system; and
sending the first data from the write queue to a second parallel file system via a beacon pulse.
13. The system of claim 12 , wherein the first data comprises information from a source of changing information.
14. The system of claim 13 , wherein the beacon pulse comprises a differential that identifies one or more changes to the changing information over an interval since a previous beacon pulse was transmitted.
15. The system of claim 12 , wherein the first parallel file system is located in a first region, and the second parallel file system is located in a second region.
16. The system of claim 15 , wherein the operations further comprise sending the first data to a third parallel file system located in a third region via the beacon pulse.
17. The system of claim 12 , wherein the operations further comprise sending additional data in the write queue to the second parallel file system via a plurality of beacon pulses.
18. A non-transitory computer-readable medium storing instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to perform operations comprising:
receiving first data from a first server;
placing the first data in a write queue associated with a first parallel file system; and
sending the first data from the write queue to a second parallel file system via a beacon pulse.
19. The non-transitory computer-readable medium of claim 18 , wherein the first data comprises information from a source of changing information, and wherein the beacon pulse comprises a differential that identifies one or more changes to the changing information over an interval since a previous beacon pulse was transmitted.
20. The non-transitory computer-readable medium of claim 18 , wherein the first parallel file system is located in a first region, the second parallel file system is located in a second region, and wherein the operations further comprise sending the first data to a third parallel file system located in a third region via the beacon pulse.