Historical generation time of data items

1 . A method for managing data storage, comprising:

receiving one or more real-time data update requests to update a plurality of data items stored at a data store, the data store configured to update the plurality of data items via a plurality of data update nodes, each data update node having a respective local map; and

in response to the one or more real-time data update requests and for each respective data item in the plurality of data items:

assigning the respective data item to a respective data update node;

updating the respective data item in the data store;

storing, in the respective local map of the respective data update node, respective supplemental data of the respective data item including a historical time stamp indicating a historical generation time of the respective data item; and

storing, in a central database, a copy of the respective supplemental data of the respective data item, the central database distinct from the respective local map and the data store; and

in response to a full data refresh request:

determining that a first messaging partition, originally assigned to a first data update node associated with the data re, is newly assigned to a second data update node associated with the data store, wherein first messaging partition is originally assigned to a first data update associated with the data store; and

for each of a set of data items of the first messaging partition,

extracting, from the central database, the copy of the respective supplemental data of the respective item; and

storing the copy of the respective supplemental data of the respective data item in a second local map associated with the second data update node.

2 . The method of claim 1 , wherein each of the plurality of data update nodes corresponds to a set of one or more messaging partitions, and each messaging partition stores data items associated with a set of predefined physical items, and wherein each data update node is configured to update data items of the set of one or more messaging partitions via real-time data updates and full data refreshes.

3 . The method of claim 1 , wherein, for each respective data item in the plurality of data items, the copy of the respective supplemental data of the respective data item is stored in the central database in response to the one or more real-time data update requests, independently of a type of the data store.

4 . The method of claim 1 , further comprising:

determining a start time of a full data refresh associated with the full data refresh request; and

for each of the set of data items of the first messaging partition, comparing the historical generation time of the respective data item with the start time, wherein in accordance with a determination that the historical generation time of the respective data item is subsequent to the start time, the copy of the respective supplemental data of the respective data item is extracted, and stored in the second local map associated with the second data update node.

5 . The method of claim 1 , further comprising, while implementing a full data refresh associated with the full data refresh request:

implementing one or more real-time data updates of a subset of data items of the first messaging partition at the second data update node; and

storing, in the second local map, the respective supplemental data of the subset of data items of the first messaging partition.

6 . The method of claim 5 , further comprising, after completing the full data refresh, consolidating the respective supplemental data of the subset of the first messaging partition stored in the first local map of the data store and the copy of the respective supplemental data extracted from the central database.

7 . The method of claim 5 , further comprising:

determining a first latest item generation time stamp corresponding to the respective supplemental data of the set of data items of the first messaging partition extracted from the central database and stored in the second local map;

determining a second latest item generation time stamp associated with the one or more real-time data updates of a subset of data items of the first messaging partition at the second data update node; and

comparing the first latest item generation time stamp and the second latest item generation time stamp.

8 . The method of claim 7 , further comprising:

in accordance with a determination that the first latest item generation time stamp is equal to or newer than the second latest item generation time stamp, repeating loading of the respective supplemental data of the set of data items of the first messaging partition from the central database to the first local map.

9 . The method of claim 5 , further comprising:

after completing the full data refresh associated with the full data refresh request, marking the first messaging partition in the second local map.

10 . The method of claim 1 , wherein for each respective data item, the respective supplemental data include the historical time stamp of the respective data item and one or more of:

a respective data item identifier uniquely identifying the respective data item;

a respective temporal range when the respective data item is updated; and

a respective messaging partition identifier uniquely identifying a respective messaging partition to which the respective data item is assigned.

11 . The method of claim 10 , further comprising:

in response to a map recovery request including a first messaging partition identifier:

identifying a first messaging partition based on the first messaging partition identifier, the first messaging partition being originally assigned to a first data update node of the data store;

extracting, from the central database, the respective supplemental data for the subset of the plurality of data items associated with the first messaging partition; and

storing, in a second local map of a second data update node, the respective supplemental data associated with the subset of the plurality of data items associated with the first messaging partition.

12 . The method of claim 10 , further comprising, in response to a map recovery request including a first data item identifier:

identifying a first data item identified by the first data item identifier;

determining that the first data item is associated with a first messaging partition that is reassigned from a first data update node to a second data update node;

extracting, from the central database, the respective supplemental data of the first data item based on the respective data item identifier; and

storing, in a second local map of the second data update node, the respective supplemental data of the first data item.

13 . The method of claim 10 , further comprising, in response to a map recovery request including a first temporal range:

determining that a subset of the plurality of data items is updated in the central database in the first temporal range;

determining that the subset of the plurality of data items is assigned to a first messaging partition newly assigned to a second data update node of the data store;

extracting, from the central database, the respective data item identifier and the historical time stamp of each of the subset of the plurality of data items; and

updating, in a second local map of the second data update node, the respective data item identifier and the historical time stamp of each of the subset of the plurality of data items.

14 . The method of claim 10 , further comprising, in response to a map recovery request including a first messaging partition identifier and a first temporal range:

identifying a subset of the plurality of data items that is generated in the first temporal range and assigned to a first messaging partition that is identified by the first messaging partition identifier;

determining that the first messaging partition is currently assigned to a second data update node; and

copying, from the central database to a second local map of the second data update node, the historical time stamp of each of the subset of the plurality of data items.

15 . A non-transitory computer-readable storage medium, having instructions stored thereon, which when executed by one or more processors cause the processors to:

receive one or more real-time data update requests to update a plurality of data items stored at a data store, the data store configured to update the plurality of data items via a plurality of data update nodes, each data update node having a respective local map; and

in response to the one or more real-time data update requests and for each respective data item in the plurality of data items:

assign the respective data item to a respective data update node;

update the respective data item in the data store;

store, in the respective local map of the respective data update node, respective supplemental data of the respective data item including a historical time stamp indicating a historical generation time of the respective data item; and

store, in a central database, a copy of the respective supplemental data of the respective data item, the central database distinct from the respective local map and the data store; and

in response to a full data refresh request:

determine that a first messaging partition, originally assigned to a first data update node associated with the data store, is newly assigned to a second data update node associated with the data store, wherein first messaging partition is originally assigned to a first data update node associated with the data store; and

for each of a set of data items of the first messaging partition:

extract, from the central database, the copy of the respective supplemental data of the respective data item; and

store the copy of the respective supplemental data of the respective data item in a second local map associated with the second data update node.

16 . The non-transitory computer-readable storage medium of claim 15 , wherein each of the plurality of data update nodes corresponds to a set of one or more messaging partitions, and each messaging partition stores data items associated with a set of predefined physical items, and wherein each data update node is configured to update data items of the set of one or more messaging partitions via real-time data updates and full data refreshes.

17 . A system, comprising:

a processor; and

a memory having instructions that when executed by the processor cause the processor to:

receive one or more data update requests to update a plurality of data items stored at a data store, the data store configured to update the plurality of data items via a plurality of data update nodes, each data update node having a respective local map; and

in response to the one or more real-time data update requests and for each respective data item in the plurality of data items:

assign the respective data item to a respective data update node;

update the respective data item in the data store;

store, in the respective local map of the respective data update node, respective supplemental data of the respective data item including a historical time stamp indicating a historical generation time of the respective data item;

store, in a central database, a copy of the respective supplemental data of the respective data item, the central database distinct from the respective local map and the data store; and

in response to a full data refresh request:

determine that a first messaging partition, originally assigned to a first data update node associated with the data store, is newly assigned to a second data update node associated with the data store, wherein first messaging partition is originally assigned to a first data update node associated with the data store; and

for each of a set of data items of the first messaging partition,

extract, from the central database, the copy of the respective supplemental data of the respective data item; and

store the copy of the respective supplemental data of the respective data item in a second local map associated with the second data update node.

18 . The system of claim 17 , wherein each of the plurality of data update nodes corresponds to a set of one or more messaging partitions, and each messaging partition stores data items associated with a set of predefined physical items, and wherein each data update node is configured to update data items of the set of one or more messaging partitions via real-time data updates and full data refreshes.

19 . The system of claim 17 , wherein, for each respective data item in the plurality of data items, the copy of the respective supplemental data of the respective data item is stored in the central database in response to the one or more real-time data update requests, independently of a type of the data store.