Distributed cache for graph data
In one embodiment, a system includes a database operative to maintain a social graph of an online social network, a leader cache layer, a plurality of servers, and a follower cache layer comprising one or more follower cache clusters, where each follower cache cluster maintains at least a portion of the social graph, and where the follower cache layer is operative to receive a command from the servers with instructions for updating a portion of the social graph, send the command to one of the leader cache layers, receive an acknowledgment of the command from one of the leader cache layers and a request to update; and update one or more of the follower cache clusters storing the portion of the social graph associated with the command.
1. A system comprising:
a database operative to maintain a social graph of an online social network, the social graph comprising a plurality of graph nodes and a plurality of graph edges connecting the graph nodes, each graph edge connecting two graph nodes indicating an association between the two graph nodes;
a leader cache layer comprising one or more leader cache clusters;
a plurality of servers, wherein the servers communicate information to and from client systems associated with users of the online social network; and
a follower cache layer comprising one or more follower cache clusters, wherein each follower cache cluster comprises one or more follower cache nodes that each maintains at least a portion of the social graph comprising the plurality of graph nodes and the plurality of graph edges connecting the graph nodes associated with the online social network, wherein each of the follower cache nodes comprises one or more individual computing systems, and wherein the follower cache layer is operative to:
receive a command from one of the plurality of servers, wherein the command comprises instructions for updating a portion of the social graph;
send the command to one of the leader cache clusters of the leader cache layer;
receive an acknowledgment of the command from one of the leader cache clusters of the leader cache layer and a request to update; and
update one or more of the follower cache clusters storing the portion of the social graph associated with the command.
2. The system of claim 1 , wherein each graph node of the social graph is associated with a unique identifier.
3. The system of claim 2 , wherein each unique identifier is stored with its respective graph node in one or more of the follower cache clusters.
4. The system of claim 1 , wherein the one or more follower cache clusters and the plurality of servers are located in close proximity.
5. The system of claim 1 , wherein each follower cache cluster of the follower cache layer is allocated a subset of a plurality of data shards, the plurality of data shards each comprising one or more shard IDs, wherein each shard ID corresponds to a particular graph node of the social graph.
6. The system of claim 5 , wherein data shards allocated to each follower cache cluster are divided among the one or more follower cache nodes of the follower cache cluster.
7. A non-transitory storage medium of a system storing computer-readable instructions, the system comprising:
a database operative to maintain a social graph of an online social network, the social graph comprising a plurality of graph nodes and a plurality of graph edges connecting the graph nodes, each graph edge connecting two graph nodes indicating an association between the two graph nodes;
a leader cache layer comprising one or more leader cache clusters;
a plurality of servers, wherein the servers communicate information to and from client systems associated with users of the online social network; and
a follower cache layer comprising one or more follower cache clusters, wherein each follower cache cluster comprises one or more follower cache nodes that each maintains at least a portion of the social graph comprising the plurality of graph nodes and the plurality of graph edges connecting the graph nodes associated with the online social network, wherein each of the follower cache nodes comprises one or more individual computing systems, and wherein the instructions, when executed, are operative to cause the follower cache layer to:
receive a command from one of the plurality of servers, wherein the command comprises instructions for updating a portion of the social graph;
send the command to one of the leader cache clusters of the leader cache layer;
receive an acknowledgment of the command from one of the leader cache clusters of the leader cache layer and a request to update; and
update one or more of the follower cache clusters storing the portion of the social graph associated with the command.
8. The media of claim 7 , wherein each graph node of the social graph is associated with a unique identifier.
9. The media of claim 8 , wherein each unique identifier is stored with its respective graph node in one or more of the one or more follower cache clusters.
10. The media of claim 7 , wherein the one or more follower cache clusters and the plurality of servers are located in close proximity.
11. The media of claim 7 , wherein each follower cache cluster of the follower cache layer is allocated a subset of a plurality of data shards, the plurality of data shards each comprising one or more shard IDs, where each shard ID corresponds to a particular graph node of the social graph.
12. The media of claim 11 , wherein data shards allocated to each follower cache cluster are divided among the one or more follower cache nodes of the follower cache cluster.
13. A method comprising:
maintaining:
a database operative to maintain a social graph of an online social network, the social graph comprising a plurality of graph nodes and a plurality of graph edges connecting the graph nodes, each graph edge connecting two graph nodes indicating an association between the two graph nodes;
a leader cache layer comprising one or more leader cache clusters;
a plurality of servers, wherein the servers communicate information to and from client systems associated with users of the online social network; and
a follower cache layer comprising one or more follower cache clusters, wherein each follower cache cluster comprises one or more follower cache nodes that each maintains at least a portion of the social graph comprising the plurality of graph nodes and the plurality of graph edges connecting the graph nodes associated with the online social network, wherein each of the follower cache nodes comprises one or more individual computing systems;
receiving, by the follower cache layer, a command from one of the plurality of servers, wherein the command comprises instructions for updating a portion of the social graph;
sending, by the follower cache layer, the command to one of the leader cache clusters of the leader cache layer;
receiving, by the follower cache layer, an acknowledgment of the command from one of the leader cache clusters of the leader cache layer and a request to update; and
updating, by the follower cache layer, one or more of the follower cache clusters storing the portion of the social graph associated with the command.
14. The method of claim 13 , wherein each graph node of the social graph is associated with a unique identifier.
15. The method of claim 14 , wherein each unique identifier is stored with its respective graph node in one or more of the one or more follower cache clusters.
16. The method of claim 13 , wherein the one or more follower cache clusters and the plurality of servers are located in close proximity.
17. The method of claim 13 , wherein each follower cache cluster of the follower cache layer is allocated a subset of a plurality of data shards, the plurality of data shards each comprising one or more shard IDs, where each shard ID corresponds to a particular graph node of the social graph.
18. The method of claim 17 , wherein data shards allocated to each follower cache cluster are divided among the one or more follower cache nodes of the follower cache cluster.