Techniques for managing data
Some techniques are described herein for tracking items across different devices. Such techniques synchronize a portion of each item while locally storing another portion. In some examples, the synchronization is performed via a conflict-free replicated data type (CRDT) and includes exchanging state vectors and/or hashes corresponding to portions of items that are being synchronized. Other techniques incrementally update local views of data as new data is received. Such techniques use storage plans that maintain operations for different local views. Other techniques manage the order of updates of different local views of data using various sets of criteria described herein. For example, some views can intentionally be left out of date while other views can be updated more frequently to satisfy requests for such views.
1. A method, comprising:
at a first process executing on a first computer system:
receiving, from a second process executing on the first computer system, a request to synchronize a first item;
identifying a first set of data of the first item, wherein the first set of data corresponds to the first computer system;
identifying a second set of data of the first item, wherein the second set of data is different from the first set of data; and
synchronizing, with a second computer system different from the first computer system, the second set of data without synchronizing the first set of data.
2. The method of claim 1 , wherein synchronizing the second set of data occurs in response to receiving, from the second computer system, a state vector that does not include an identifier of the second set of data.
3. The method of claim 1 , wherein synchronizing the second set of data includes sending, to the second computer system, a hash of the second set of data.
4. The method of claim 1 , wherein synchronizing the second set of data with the second computer system includes:
receiving, from the second computer system, a request for the second set of data; and
in response to receiving the request for the second set of data, sending, to the second computer system, the second set of data.
5. The method of claim 1 , further comprising:
as part of synchronizing with the second computer system:
sending, to the second computer system, a state vector for synchronized items known by the first process;
receiving, from the second computer system, one or more hash values;
in response to receiving the one or more hash values:
in accordance with a determination that a first hash value of the one or more hash values is not known by the first process, sending a request for data corresponding to the first hash value; and
in accordance with a determination that a second hash value of the one or more hash values is known by the first process, updating an object corresponding to the second hash value to include a computer system corresponding to the second hash value; and
after sending the request for the data corresponding to the first hash value, receiving the data corresponding to the first hash value.
6. The method of claim 1 , wherein the first item is an item of a first type, the method further comprising:
receiving a request to synchronize a second item of a second type different from the first type;
identifying a third set of data of the second item, wherein the third set of data is specific to the first computer system;
identifying a fourth set of data of the second item, wherein the fourth set of data is not specific to the first computer system, and wherein the fourth set of data is different from the third set of data; and
synchronizing, with the second computer system, the fourth set of data without synchronizing the third set of data.
7. The method of claim 6 , wherein the request to synchronize the second item is received from a third process executing on the first computer system, and wherein the third process is different from the first process and the second process.
8. The method of claim 1 , further comprising:
after synchronizing the second set of data, receiving, from a fourth process executing on the first computer system, a first request corresponding to the first item, wherein the fourth process is different from the first process; and
in response to receiving the first request corresponding to the first item, sending to the fourth process, a first state vector including an indication of the first set of data and a second state vector including an indication of the second set of data, wherein the second state vector is different from the first state vector.
9. The method of claim 1 , further comprising:
after synchronizing the second set of data with the second computer system, receiving, from a fifth process executing on the first computer system, a second request corresponding to the first item, wherein the fifth process is different from the first process; and
in response to receiving the second request corresponding to the first item:
in accordance with a determination that the second computer system includes an item with data matching the second set of data, sending, to the fifth process, an indication of the first computer system and an indication of the second computer system; and
in accordance with a determination that the second computer system does not include an item with data matching the second set of data, sending, to the fifth process, the indication of the first computer system without the indication of the second computer system.
10. The method of claim 1 , further comprising:
after synchronizing the second set of data with the second computer system, receiving, from a sixth process executing on the first computer system, a third request corresponding to the first item, wherein the sixth process is different from the first process; and
in response to receiving the third request corresponding to the first item, sending to the sixth process, the first set of data.
11. The method of claim 1 , further comprising:
after synchronizing the second set of data with the second computer system, receiving, from the second process, an indication of an update to the first item; and
in response to receiving the indication of the update to the first item:
in accordance with a determination that the update at least partially corresponds to data not specific to the first computer system:
updating the second set of data; and
synchronizing, with the second computer system, the updated second set of data without synchronizing data specific to the first computer system; and
in accordance with a determination that the update at least partially corresponds to data specific to the first computer system:
updating the first set of data; and
forgoing synchronizing, with the second computer system, the updated first set of data.
12. The method of claim 1 , further comprising:
after synchronizing the second set of data with the second computer system, receiving, from the second process, an indication of a current state of the first item; and
after receiving the current state of the first item, comparing the current state of the first item with a previous state of the first item; and
in response to comparing the current state of the first item with the previous state of the first item:
in accordance with a determination that data of the first item not specific to the first computer system has been updated:
updating the second set of data to generate an updated second set of data; and
synchronizing, with the second computer system, the updated second set of data without synchronizing data specific to the first computer system; and
in accordance with a determination that data of the first item specific to the first computer system has been updated:
updating the first set of data to generate an updated first set of data; and
forgoing synchronizing, with the second computer system, the updated first set of data.
13. The method of claim 1 , wherein the first set of data is specific to the first computer system and wherein the second data is not specific to the first computer system.
14. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first computer system executing a first process, the one or more programs including instructions for:
receiving, from a second process executing on the first computer system, a request to synchronize a first item;
identifying a first set of data of the first item, wherein the first set of data corresponds to the first computer system;
identifying a second set of data of the first item, wherein the second set of data is different from the first set of data; and
synchronizing, with a second computer system different from the first computer system, the second set of data without synchronizing the first set of data.
15. The non-transitory computer-readable storage medium of claim 14 , wherein synchronizing the second set of data occurs in response to receiving, from the second computer system, a state vector that does not include an identifier of the second set of data.
16. The non-transitory computer-readable storage medium of claim 14 , wherein synchronizing the second set of data includes sending, to the second computer system, a hash of the second set of data.
17. The non-transitory computer-readable storage medium of claim 14 , wherein synchronizing the second set of data with the second computer system includes:
receiving, from the second computer system, a request for the second set of data; and
in response to receiving the request for the second set of data, sending, to the second computer system, the second set of data.
18. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more programs further include instructions for:
as part of synchronizing with the second computer system:
sending, to the second computer system, a state vector for synchronized items known by the first process;
receiving, from the second computer system, one or more hash values;
in response to receiving the one or more hash values:
in accordance with a determination that a first hash value of the one or more hash values is not known by the first process, sending a request for data corresponding to the first hash value; and
in accordance with a determination that a second hash value of the one or more hash values is known by the first process, updating an object corresponding to the second hash value to include a computer system corresponding to the second hash value; and
after sending the request for the data corresponding to the first hash value, receiving the data corresponding to the first hash value.
19. The non-transitory computer-readable storage medium of claim 14 , wherein the first item is an item of a first type, wherein the one or more programs further include instructions for:
receiving a request to synchronize a second item of a second type different from the first type;
identifying a third set of data of the second item, wherein the third set of data is specific to the first computer system;
identifying a fourth set of data of the second item, wherein the fourth set of data is not specific to the first computer system, and wherein the fourth set of data is different from the third set of data; and
synchronizing, with the second computer system, the fourth set of data without synchronizing the third set of data.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the request to synchronize the second item is received from a third process executing on the first computer system, and wherein the third process is different from the first process and the second process.
21. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more programs further include instructions for:
after synchronizing the second set of data, receiving, from a fourth process executing on the first computer system, a first request corresponding to the first item, wherein the fourth process is different from the first process; and
in response to receiving the first request corresponding to the first item, sending to the fourth process, a first state vector including an indication of the first set of data and a second state vector including an indication of the second set of data, wherein the second state vector is different from the first state vector.
22. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more programs further include instructions for:
after synchronizing the second set of data with the second computer system, receiving, from a fifth process executing on the first computer system, a second request corresponding to the first item, wherein the fifth process is different from the first process; and
in response to receiving the second request corresponding to the first item:
in accordance with a determination that the second computer system includes an item with data matching the second set of data, sending, to the fifth process, an indication of the first computer system and an indication of the second computer system; and
in accordance with a determination that the second computer system does not include an item with data matching the second set of data, sending, to the fifth process, the indication of the first computer system without the indication of the second computer system.
23. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more programs further include instructions for:
after synchronizing the second set of data with the second computer system, receiving, from a sixth process executing on the first computer system, a third request corresponding to the first item, wherein the sixth process is different from the first process; and
in response to receiving the third request corresponding to the first item, sending to the sixth process, the first set of data.
24. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more programs further include instructions for:
after synchronizing the second set of data with the second computer system, receiving, from the second process, an indication of an update to the first item; and
in response to receiving the indication of the update to the first item:
in accordance with a determination that the update at least partially corresponds to data not specific to the first computer system:
updating the second set of data; and
synchronizing, with the second computer system, the updated second set of data without synchronizing data specific to the first computer system; and
in accordance with a determination that the update at least partially corresponds to data specific to the first computer system:
updating the first set of data; and
forgoing synchronizing, with the second computer system, the updated first set of data.
25. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more programs further include instructions for:
after synchronizing the second set of data with the second computer system, receiving, from the second process, an indication of a current state of the first item; and
after receiving the current state of the first item, comparing the current state of the first item with a previous state of the first item; and
in response to comparing the current state of the first item with the previous state of the first item:
in accordance with a determination that data of the first item not specific to the first computer system has been updated:
updating the second set of data to generate an updated second set of data; and
synchronizing, with the second computer system, the updated second set of data without synchronizing data specific to the first computer system; and
in accordance with a determination that data of the first item specific to the first computer system has been updated:
updating the first set of data to generate an updated first set of data; and
forgoing synchronizing, with the second computer system, the updated first set of data.
26. The non-transitory computer-readable storage medium of claim 14 , wherein the first set of data is specific to the first computer system and wherein the second data is not specific to the first computer system.
27. A first computer system, comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for a first process to:
receive, from a second process executing on the first computer system, a request to synchronize a first item;
identifying a first set of data of the first item, wherein the first set of data corresponds to the first computer system;
identifying a second set of data of the first item, wherein the second set of data is different from the first set of data; and
synchronize, with a second computer system different from the first computer system, the second set of data without synchronizing the first set of data.
28. The first computer system of claim 27 , wherein synchronizing the second set of data occurs in response to receiving, from the second computer system, a state vector that does not include an identifier of the second set of data.
29. The first computer system of claim 27 , wherein synchronizing the second set of data includes sending, to the second computer system, a hash of the second set of data.
30. The first computer system of claim 27 , wherein synchronizing the second set of data with the second computer system includes:
receiving, from the second computer system, a request for the second set of data; and
in response to receiving the request for the second set of data, sending, to the second computer system, the second set of data.
31. The first computer system of claim 27 , wherein the one or more programs further include instructions for the first process to:
as part of synchronizing with the second computer system:
send, to the second computer system, a state vector for synchronized items known by the first process;
receive, from the second computer system, one or more hash values;
in response to receiving the one or more hash values:
in accordance with a determination that a first hash value of the one or more hash values is not known by the first process, send a request for data corresponding to the first hash value; and
in accordance with a determination that a second hash value of the one or more hash values is known by the first process, update an object corresponding to the second hash value to include a computer system corresponding to the second hash value; and
after sending the request for the data corresponding to the first hash value, receive the data corresponding to the first hash value.
32. The first computer system of claim 27 , wherein the first item is an item of a first type, wherein the one or more programs further include instructions for:
receive a request to synchronize a second item of a second type different from the first type;
identify a third set of data of the second item, wherein the third set of data is specific to the first computer system;
identify a fourth set of data of the second item, wherein the fourth set of data is not specific to the first computer system, and wherein the fourth set of data is different from the third set of data; and
synchronize, with the second computer system, the fourth set of data without synchronizing the third set of data.
33. The first computer system of claim 32 , wherein the request to synchronize the second item is received from a third process executing on the first computer system, and wherein the third process is different from the first process and the second process.
34. The first computer system of claim 27 , wherein the one or more programs further include instructions for the first process to:
after synchronizing the second set of data, receive, from a fourth process executing on the first computer system, a first request corresponding to the first item, wherein the fourth process is different from the first process; and
in response to receiving the first request corresponding to the first item, send to the fourth process, a first state vector including an indication of the first set of data and a second state vector including an indication of the second set of data, wherein the second state vector is different from the first state vector.
35. The first computer system of claim 27 , wherein the one or more programs further include instructions for the first process to:
after synchronizing the second set of data with the second computer system, receive, from a fifth process executing on the first computer system, a second request corresponding to the first item, wherein the fifth process is different from the first process; and
in response to receiving the second request corresponding to the first item:
in accordance with a determination that the second computer system includes an item with data matching the second set of data, send, to the fifth process, an indication of the first computer system and an indication of the second computer system; and
in accordance with a determination that the second computer system does not include an item with data matching the second set of data, send, to the fifth process, the indication of the first computer system without the indication of the second computer system.
36. The first computer system of claim 27 , wherein the one or more programs further include instructions for the first process to:
after synchronizing the second set of data with the second computer system, receive, from a sixth process executing on the first computer system, a third request corresponding to the first item, wherein the sixth process is different from the first process; and
in response to receiving the third request corresponding to the first item, send to the sixth process, the first set of data.
37. The first computer system of claim 27 , wherein the one or more programs further include instructions for the first process to:
after synchronizing the second set of data with the second computer system, receive, from the second process, an indication of an update to the first item; and
in response to receiving the indication of the update to the first item:
in accordance with a determination that the update at least partially corresponds to data not specific to the first computer system:
update the second set of data; and
synchronize, with the second computer system, the updated second set of data without synchronizing data specific to the first computer system; and
in accordance with a determination that the update at least partially corresponds to data specific to the first computer system:
update the first set of data; and
forgo synchronizing, with the second computer system, the updated first set of data.
38. The first computer system of claim 27 , wherein the one or more programs further include instructions for the first process to:
after synchronizing the second set of data with the second computer system, receive, from the second process, an indication of a current state of the first item; and
after receiving the current state of the first item, compare the current state of the first item with a previous state of the first item; and
in response to comparing the current state of the first item with the previous state of the first item:
in accordance with a determination that data of the first item not specific to the first computer system has been updated:
update the second set of data to generate an updated second set of data; and
synchronize, with the second computer system, the updated second set of data without synchronizing data specific to the first computer system; and
in accordance with a determination that data of the first item specific to the first computer system has been updated:
update the first set of data to generate an updated first set of data; and
forgo synchronizing, with the second computer system, the updated first set of data.
39. The first computer system of claim 27 , wherein the first set of data is specific to the first computer system and wherein the second data is not specific to the first computer system.