Unobtrusive copies of actively used compressed indices
View Patent ↗Methods, devices and systems to make compressed backup copies of in-use compressed database indices are described. In general, an “oldest” time at which index pages in working memory had been updated is identified. Compressed index pages may be directly copied without the need to bring them into working memory or uncompressing them. The identified “oldest” time is then associated with the compressed backup copy. In some embodiments, an entire compressed backup copy may be associated with a single point in time (e.g., the identified “oldest” time). In other embodiments, a compressed backup copy may be associated with multiple points in time (e.g., one time for each portion of the compressed index that is being backed-up). Compressed indices copied in accordance with the invention may be used during restore operations to reconstruct database indices using the identified “oldest” time and database log files.
1. A general purpose computer comprising:
at least one processor; and
at least one storage device tangibly embodying program instructions that, when executed by the at least one processor, are configured to cause the general purpose computer to at least:
for each of a plurality of pages associated with a target index:
write the page to external storage; and
store a time of the writing the page to the external storage as an externalization time of the page; and
after performing the writing and the storing:
determine which pages in a buffer memory are associated with the target index;
identify an externalization point of the target index, the externalization point corresponding to an oldest change to a page associated with the target index that has not been written to the external storage;
for each of the pages associated with the target index that is stored in the buffer memory, if the externalization point of the target index is earlier than the externalization time of the page, write the page from the buffer memory to an output file; and
for each of the pages associated with the target index that is not stored in the buffer memory, and each of the pages associated with the target index and stored in the buffer memory for which the externalization point of the target index is later than the externalization time of the page, copy the page from the external storage to the output file without bringing the page into the buffer memory.
2. The general purpose computer of claim 1 , wherein the writing the page to the external storage and the storing the time of the writing the page to the external storage are performed without employing locks on the target index.
3. The general purpose computer of claim 1 , wherein the program instructions are further configured to cause the general purpose computer to compress the pages using a technique wherein the compression of a first page of the target index does not affect content of a second page of the target index.
4. The general purpose computer of claim 1 , wherein the target index comprises an encrypted compressed index.
5. The general purpose computer of claim 1 , wherein the identifying the externalization point of the target index comprises:
identifying one or more pages of the target index that are in a working memory, uncompressed and accessible to the general purpose computer;
determining the oldest change to the page of the target index that is in the working memory, uncompressed and accessible to the general purpose computer; and
identifying the determined oldest change as the externalization point.
6. The general purpose computer of claim 1 , wherein the storing the time of the writing the page comprises storing a single time of the writing the page to the external storage as the externalization time of the page.
7. The general purpose computer of claim 1 , wherein the storing the time of the writing the page comprises storing a plurality of times of the writing the page to the external storage as externalization times of the page, wherein each of the plurality of externalization times represents an externalization time for a different portion of the page.
8. The general purpose computer of claim 1 , wherein the storing is performed substantially independent of the general purpose computer.
9. The general purpose computer of claim 1 , wherein the storing the time of the writing as the externalization time of the page comprises:
recording an identifier for the stored page;
recording the externalization time; and
associating the recorded externalization time with the recorded identifier for the stored page.
10. The general purpose computer of claim 9 , wherein the associating the recorded externalization time with the recorded identifier comprises recording the identifier for the stored page and the externalization time in a common database record.
11. The general purpose computer of claim 1 , wherein the externalization time is identified by an address representing an entry in a log file corresponding to a time at which the page was modified.
12. A non-transitory program storage device, readable by a programmable control device, comprising program instructions stored on the non-transitory program storage device for causing the programmable control device to perform at least the following:
for each of a plurality of pages associated with a target index:
write the page to external storage; and
store a time of the writing the page to the external storage as an externalization time of the page; and
after performing the writing and the storing:
determine which pages in a buffer memory are associated with the target index;
identify an externalization point of the target index, the externalization point corresponding to an oldest change to a page associated with the target index that has not been written to the external storage;
for each of the pages associated with the target index that is stored in the buffer memory, if the externalization point of the target index is earlier than the externalization time of the page, write the page from the buffer memory to an output file; and
for each of the pages associated with the target index that is not stored in the buffer memory, and each of the pages associated with the target index and stored in the buffer memory for which the externalization point of the target index is later than the externalization time of the page, copy the page from the external storage to the output file without bringing the page into the buffer memory.
13. A computer network executing a method to backup a target index, the computer network comprising:
a first storage device having stored thereon at least a portion of the target index, the first storage device being external to a computer system;
the computer system communicatively coupled to the first storage device, the computer system having memory, the memory having stored therein at least one page of the target index and program instructions for causing the computer system to perform at least the following:
for each of a plurality of pages associated with the target index:
write the page to the first storage device; and
store a time of the writing the page to the first storage device as an externalization time of the page; and
after performing the writing and the storing:
determine which pages in a buffer memory are associated with the target index;
identify an externalization point of the target index, the externalization point corresponding to an oldest change to a page associated with the target index that has not been written to the first storage device;
for each of the pages associated with the target index that is stored in the buffer memory, if the externalization point of the target index is earlier than the externalization time of the page, write the page from the buffer memory to a second storage device; and
for each of the pages associated with the target index that is not stored in the buffer memory, and each of the pages associated with the target index and stored in the buffer memory for which the externalization point of the target index is later than the externalization time of the page, copy the page from the first storage device to the second storage device without bringing the page into the buffer memory.