Using a tree structure to segment and distribute records across one or more decentralized, acyclic graphs of cryptographic hash pointers
Provided is a process including: receiving, with one or more processors, a first request to store a record from a computing entity; encoding, with one or more processors, the record in a first plurality of segments; arranging, with one or more processors, the first plurality of segments in respective content nodes of a first content graph, wherein at least some content nodes of the first content graph have two or more content edges of the first content graph pointing to two or more respective other content nodes of the first content graph; and storing, with one or more processors, the content nodes of the first content graph in a verification graph.
1. A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors effectuate operations configured to expedite retrieval of records written to persistent storage, the operations comprising:
receiving, with one or more processors, a request to read a record from a computing entity, the request including an identifier of a root content node, wherein the record is a file, the identifier distinguishes the root content node from other content nodes of a content graph;
retrieving, with one or more processors and using the identifier, the root content node from a verification graph, wherein:
the verification graph comprises a plurality of verification nodes,
the content nodes and content edges are stored as content of the verification nodes,
respective pairs of the verification nodes are connected by respective directed verification edges, and
the verification graph is configured to indicate tampering with any of a plurality of records stored in the verification graph;
traversing, with one or more processors, the content graph from the root content node to retrieve content nodes of the content graph from the verification graph, wherein at least some of the content nodes are retrieved concurrently, wherein the verification graph includes a plurality of directed acyclic graphs of cryptographic hash pointers and different portions of the content graph are stored as content of verification nodes of the directed acyclic graphs of cryptographic hash pointers;
verifying, with one or more processors, that each segment of a plurality of segments stored in the every content node are consistent with the verification graph; and
returning, with one or more processors and in response to the verifying that the segments are consistent with the verification graph, the record according to the request.
2. The tangible, non-transitory, machine-readable medium of claim 1 , wherein the operations further comprise:
signaling, with one or more processors and in response to the verifying that the segments are not consistent with the verification graph, that the record has been compromised.
3. The tangible, non-transitory, machine-readable medium of claim 2 , wherein the verifying that the segments are not consistent with the verification graph includes detecting an inconsistent cryptograph hash pointer within the verification graph.
4. The tangible, non-transitory, machine-readable medium of claim 1 , wherein the verifying that the segments are consistent with the verification graph includes detecting that the cryptograph hash pointers within the verification graph are consistent.
5. The tangible, non-transitory, machine-readable medium of claim 1 , wherein the returning the record according to the request includes:
decoding the segments into the record.
6. The tangible, non-transitory, machine-readable medium of claim 5 , wherein the decoding the segments into the record includes concatenating the segments.
7. The tangible, non-transitory, machine-readable medium of claim 5 , wherein the decoding the segments into the record includes XOR'ing the segments to re-create the record.
8. The tangible, non-transitory, machine-readable medium of claim 1 , wherein:
the content graph includes a binary tree that accounts for at least half of the content nodes of the content graph.
9. The tangible, non-transitory, machine-readable medium of claim 8 , wherein:
the content graph is a balanced binary tree.
10. The tangible, non-transitory, machine-readable medium of claim 1 , wherein:
the content graph includes a skip list defined by the content edges of the content graph.
11. The tangible, non-transitory, machine-readable medium of claim 1 , wherein:
the content graph defines k-ary tree in which each content node of the k-ary tree has fewer k or fewer child content nodes, wherein k is 3 or larger.
12. The tangible, non-transitory, machine-readable medium of claim 11 , wherein:
the content graph includes three or more levels of hierarchy.
13. The tangible, non-transitory, machine-readable medium of claim 11 , wherein:
at least half of the content nodes of the k-ary tree have 2 or more child nodes.
14. The tangible, non-transitory, machine-readable medium of claim 11 , wherein the operations comprise:
receiving a second request to store the record;
encoding the record in the plurality of segments;
arranging the plurality of segments in respective content nodes of the content graph;
storing the content nodes of the content graph in the verification graph,
returning the identifier of the root content node to the computing entity, wherein the identifier identifies a verification node of the verification graph storing the root content node.
15. The tangible, non-transitory, machine-readable medium of claim 14 , wherein:
each segment is stored in a different content node of the content graph;
directed content edges of the content graph connect respective pairs of the content nodes of the content graph;
the directed content edges define a plurality of directed paths through the content graph by which each of the segments is reachable from the root content node of the content graph; and
at least some content nodes of the content graph have two or more content edges of the content graph pointing to two or more respective other content nodes of the content graph.
16. The tangible, non-transitory, machine-readable medium of claim 1 , wherein:
the content graph comprises means for storing segmented data to facilitate concurrent retrieval.
17. The tangible, non-transitory, machine-readable medium of claim 1 , wherein:
the request is a request to read the file from the persistent storage from a security driver of an operating system of a user computing device.
18. The tangible, non-transitory, machine-readable medium of claim 1 , wherein:
read latency of the record from the verification graph scales at less than O(n) in big O notation, where n is a number of the segments; and
traversing the content graph comprises performing a breadth first traversal of at least part of the verification graph.
19. A method to expedite retrieval of records written to persistent storage, comprising:
receiving, with one or more processors, a request to read a record from a computing entity, the request including an identifier of a root content node, wherein the record is a file, the identifier distinguishes the root content node from other content nodes of a content graph;
retrieving, with one or more processors and using the identifier, the root content node from a verification graph, wherein:
the verification graph comprises a plurality of verification nodes,
the content nodes and content edges are stored as content of the verification nodes, respective pairs of the verification nodes are connected by respective directed verification edges, and
the verification graph is configured to indicate tampering with any of a plurality of records stored in the verification graph;
traversing, with one or more processors, the content graph from the root content node to retrieve content nodes of the content graph from the verification graph, wherein at least some of the content nodes are retrieved concurrently, wherein the verification graph includes a plurality of directed acyclic graphs of cryptographic hash pointers and different portions of the content graph are stored as content of verification nodes of the directed acyclic graphs of cryptographic hash pointers;
verifying, with one or more processors, that each segment of a plurality of segments stored in the every content node are consistent with the verification graph; and
returning, with one or more processors and in response to the verifying that the segments are consistent with the verification graph, the record according to the request.
20. The method of claim 19 , comprising:
signaling, with one or more processors and in response to the verifying that the segments are not consistent with the verification graph, that the record has been compromised.
21. The method of claim 20 , wherein the verifying that the segments are not consistent with the verification graph includes detecting an inconsistent cryptograph hash pointer within the verification graph.
22. The method of claim 19 , wherein the verifying that the segments are consistent with the verification graph includes detecting that the cryptograph hash pointers within the verification graph are consistent.
23. The method of claim 19 , wherein the returning the record according to the request includes:
decoding the segments into the record.
24. The method of claim 23 , wherein the decoding the segments into the record includes concatenating the segments.
25. The method of claim 23 , wherein the decoding the segments into the record includes XOR'ing the segments to re-create the record.
26. The method of claim 19 , wherein:
the content graph includes a binary tree that accounts for at least half of the content nodes of the content graph.
27. The method of claim 19 , wherein:
the content graph is a balanced binary tree.
28. The method of claim 19 , wherein:
the content graph includes a skip list defined by the content edges of the content graph.
29. The method of claim 19 , wherein:
the content graph defines k-ary tree in which each content node of the k-ary tree has fewer k or fewer child content nodes, wherein k is 3 or larger.
30. The method of claim 29 , wherein:
the content graph includes three or more levels of hierarchy.
31. The method of claim 29 , wherein:
at least half of the content nodes of the k-ary tree have 2 or more child nodes.
32. The method of claim 29 , further comprising:
receiving a second request to store the record;
encoding the record in the plurality of segments;
arranging the plurality of segments in respective content nodes of the content graph;
storing the content nodes of the content graph in the verification graph,
returning the identifier of the root content node to the computing entity, wherein the identifier identifies a verification node of the verification graph storing the root content node.
33. The method of claim 32 , wherein:
each segment is stored in a different content node of the content graph;
directed content edges of the content graph connect respective pairs of the content nodes of the content graph;
the directed content edges define a plurality of directed paths through the content graph by which each of the segments is reachable from the root content node of the content graph; and
at least some content nodes of the content graph have two or more content edges of the content graph pointing to two or more respective other content nodes of the content graph.
34. The method of claim 19 , wherein:
the content graph comprises means for storing segmented data to facilitate concurrent retrieval.
35. The method of claim 19 , wherein:
the request is a request to read the file from the persistent storage from a security driver of an operating system of a user computing device.
36. The method of claim 19 , wherein:
read latency of the record from the verification graph scales at less than 0 (n) in big O notation, where n is a number of the segments; and
traversing the content graph comprises performing a breadth first traversal of at least part of the verification graph.