System and method for data storage
In one embodiment, a method includes storing a first data table entry applying a schema used at a first point in time along with pointer data and pattern space information collectively representing the first data table entry at the first point in time and generating, at a later point in time, a map responsive to a schema change. The map comprises new pointer data or new pattern space information.
1. A method comprising:
by one or more computing devices, receiving a query corresponding to a first schema of a first data extract associated with a pattern space;
by the one or more computing devices, accessing a second data extract associated with the pattern space, the second data extract having a second schema that is different from the first schema;
by the one or more computing devices, mapping the second schema of the second data extract to the first schema of the first data extract; and
by the one or more computing devices, running the query against the second data extract based on the mapping;
wherein:
the first data extract comprises a first header portion, a first index portion, and a first pointer to the pattern space;
the second data extract comprises a second header portion, a second index portion, and a second pointer to the pattern space;
the second index portion is the same as the first index portion;
the second pointer to the pattern space is the same as the first pointer to the pattern space;
the first header portion comprises first metadata associated with the first data extract;
the second header portion comprises second metadata associated with the second data extract; and
the mapping is based at least in part on the first header portion.
2. The method of claim 1 , wherein the first metadata comprises names, types, and attributes of one or more first fields, and the second metadata comprises names, types, and attributes of one or more second fields.
3. The method of claim 2 , wherein:
the first metadata comprises first path information associated with a particular one of the one or more first fields;
the second metadata comprises second path information associated with a particular one of the one or more second fields; and
the mapping is based at least in part on the first path information.
4. The method of claim 3 , wherein the first path information indicates an entry in the first index portion that points to one or more patterns in the pattern space.
5. The method of claim 3 , wherein the first path information comprises one or more functions.
6. The method of claim 2 , wherein the one or more first fields comprise one or more, but not all, of the one or more second fields.
7. The method of claim 2 , wherein the one or more second fields comprise one or more, but not all, of the one or more first fields.
8. The method of claim 2 , wherein the one or more first fields comprise a renamed field of the one or more second fields.
9. The method of claim 1 , wherein the first header portion comprises a pointer to the second header portion.
10. The method of claim 1 , wherein the first data extract and the second data extract each comprise immutable data.
11. The method of claim 2 , wherein the one or more first fields comprise a re-typed field of the one or more second fields.
12. One or more computer-readable non-transitory storage media embodying software that is configured when executed to:
receive a query corresponding to a first schema of a first data extract associated with a pattern space;
access a second data extract associated with the pattern space, the second data extract having a second schema that is different from the first schema;
map the second schema of the second data extract to the first schema of the first data extract; and
run the query against the second data extract based on the mapping;
wherein:
the first data extract comprises a first header portion, a first index portion, and a first pointer to the pattern space;
the second data extract comprises a second header portion, a second index portion, and a second pointer to the pattern space;
the second index portion is the same as the first index portion;
the second pointer to the pattern space is the same as the first pointer to the pattern space;
the first header portion comprises first metadata associated with the first data extract;
the second header portion comprises second metadata associated with the second data extract; and
the mapping is based at least in part on the first header portion.
13. The one or more computer-readable non-transitory storage media of claim 12 , wherein the first metadata comprises names, types, and attributes of one or more first fields, and the second metadata comprises names, types, and attributes of one or more second fields.
14. The one or more computer-readable non-transitory storage media of claim 13 , wherein:
the first metadata comprises first path information associated with a particular one of the one or more first fields;
the second metadata comprises second path information associated with a particular one of the one or more second fields; and
the mapping is based at least in part on the first path information.
15. The one or more computer-readable non-transitory storage media of claim 14 , wherein the first path information indicates an entry in the first index portion that points to one or more patterns in the pattern space.
16. The one or more computer-readable non-transitory storage media of claim 14 , wherein the first path information comprises one or more functions.
17. The one or more computer-readable non-transitory storage media of claim 13 , wherein the one or more first fields comprise one or more, but not all, of the one or more second fields.
18. The one or more computer-readable non-transitory storage media of claim 13 , wherein the one or more second fields comprise one or more, but not all, of the one or more first fields.
19. The one or more computer-readable non-transitory storage media of claim 13 , wherein the one or more first fields comprise a renamed field of the one or more second fields.
20. The one or more computer-readable non-transitory storage media of claim 12 , wherein the first header portion comprises a pointer to the second header portion.
21. The one or more computer-readable non-transitory storage media of claim 12 , wherein the first data extract and the second data extract each comprise immutable data.
22. The one or more computer-readable non-transitory storage media of claim 13 , wherein the one or more first fields comprise a re-typed field of the one or more second fields.
23. A system comprising:
one or more computing devices, the one or more computing devices comprising one or more processors; and
a memory coupled to the one or more processors comprising instructions executable by the one or more processors, the one or more processors configured when executing the instructions to:
receive a query corresponding to a first schema of a first data extract associated with a pattern space;
access a second data extract associated with the pattern space, the second data extract having a second schema that is different from the first schema;
map the second schema of the second data extract to the first schema of the first data extract; and
run the query against the second data extract based on the mapping;
wherein:
the first data extract comprises a first header portion, a first index portion, and a first pointer to the pattern space;
the second data extract comprises a second header portion, a second index portion, and a second pointer to the pattern space;
the second index portion is the same as the first index portion;
the second pointer to the pattern space is the same as the first pointer to the pattern space;
the first header portion comprises first metadata associated with the first data extract;
the second header portion comprises second metadata associated with the second data extract; and
the mapping is based at least in part on the first header portion.
24. The system of claim 23 , wherein the first metadata comprises names, types, and attributes of one or more first fields, and the second metadata comprises names, types, and attributes of one or more second fields.
25. The system of claim 24 , wherein:
the first metadata comprises first path information associated with a particular one of the one or more first fields;
the second metadata comprises second path information associated with a particular one of the one or more second fields; and
the mapping is based at least in part on the first path information.
26. The system of claim 25 , wherein the first path information indicates an entry in the first index portion that points to one or more patterns in the pattern space.
27. The system of claim 25 , wherein the first path information comprises one or more functions.
28. The system of claim 24 , wherein the one or more first fields comprise one or more, but not all, of the one or more second fields.
29. The system of claim 24 , wherein the one or more second fields comprise one or more, but not all, of the one or more first fields.
30. The system of claim 24 , wherein the one or more first fields comprise a renamed field of the one or more second fields.
31. The system of claim 24 , wherein the one or more first fields comprise a re-typed field of the one or more second fields.
32. The system of claim 23 , wherein the first header portion comprises a pointer to the second header portion.