Saving snapshot of a knowledge base without blocking
A consistent snapshot of a large main memory knowledge base is saved to persistent storage without blocking the application for the duration of serializing and writing the knowledge base. Taking the snapshot comprises bringing the knowledge base to a consistent state (in a multithreaded application), using virtual memory facilities to obtain a copy-on-write copy of the knowledge base in memory, and using a separate thread or process to serialize the copy-on-write copy to persistent storage.
1. A method for saving a snapshot of a large knowledge base comprising a cyclic data structure without blocking, comprising:
bringing, using a consistent state means in a computer, the knowledge base to a consistent state;
creating, using a copy-on-write means in a computer, a copy-on-write virtual copy of the knowledge base and thereafter allowing mutators to continue modifying the knowledge base; and
serializing, using a serializing means in a computer, the virtual copy of the knowledge base into a snapshot of the knowledge base in persistent storage without blocking accesses or updates to the knowledge base, and in response to completing the snapshot, freeing the virtual copy of the knowledge base.
2. The method of claim 1 , further comprising:
while the serializing is being performed, making updates to the knowledge base from other threads or processes.
3. The method of claim 1 , wherein bringing the knowledge base to a consistent state involves taking a global lock that is used to synchronize updates to the knowledge base during transaction commits.
4. The method of claim 1 , wherein creating a copy-on-write virtual copy of the knowledge base comprises using the fork function.
5. The method of claim 1 , wherein creating a copy-on-write virtual copy of the knowledge base comprises calling the mmap function one or more times.
6. The method of claim 1 , wherein creating a copy-on-write virtual copy of the knowledge base comprises calling the MapViewOfFile function one or more times.
7. The method of claim 1 , wherein the serialization is performed using a serialization method capable of handling object graphs that comprise cyclic data.
8. The method of claim 1 , wherein the serialization is performed using the built-in serialization method in Java.
9. The method of claim 1 , wherein the serializing comprises adjusting pointers so that they point to the correct address in the virtual copy.
10. A computer comprising:
one or more processors ( 101 ), with at least one of the processors equipped with a virtual memory means ( 102 );
a consistent state means ( 109 );
a copy-on-write means ( 108 ) configures to make use of the virtual memory means to create a virtual copy of a knowledge base comprising a cyclic data structure
brought into a consistent state using the consistent state means before allowing mutators to continue modifying the knowledge base; and
a serializing means ( 110 ) for serializing the virtual copy of the knowledge base into a snapshot of the knowledge base in persistent storage without blocking access or updates to the knowledge base before freeing the virtual copy of the knowledge base.
11. A computer program product stored on a non-transitory computer readable medium operable to cause a computer to save a snapshot of a large main memory knowledge base comprising a cyclic data structue without blocking, comprising:
computer usable program code means for bringing the knowledge base to a consistent state;
computer usable program code means for creating a copy-on-write virtual copy of the knowledge base brought to a consistent state and thereafter allowing mutators to continue modifying the knowledge bas; and
computer usable program code means for serializing the virtual copy of the knowledge base into a snapshot of the knowledge base in persistent storage without blocking accesses or updates to the knowledge base, and in response to completing the snapshot, freeing the virtual copy of the knowledge base.