Supporting piecewise update of JSON document efficiently
Herein are fine grained updates to pieces of JavaScript object notation (JSON) documents by database statements that can update, delete, and insert parts of JSON documents. In an embodiment, a computer receives a request that specifies a modification of a JSON document that is stored in a compressed format in persistent storage. The modification adds additional content to the JSON document, and/or replaces an old value in the JSON document with a new value that is not a same size as the old value. The modification is recorded in a change log. The change log is eventually applied to the compressed format of the JSON document in the persistent storage without entirely rewriting the compressed format of the JSON document in the persistent storage.
1. A method comprising:
storing a JavaScript object notation (JSON) document in a compressed format in persistent storage;
automatically deciding, after said storing the JSON document in the compressed format in persistent storage, to compact the JSON document in the compressed format in persistent storage;
compacting, in response to said automatically deciding to compact the JSON document in the compressed format in persistent storage, the JSON document in the compressed format in persistent storage.
2. The method of claim 1 wherein:
said storing the JSON document in the compressed format in persistent storage comprises inlining the JSON document in the compressed format in a table row in persistent storage;
the method further comprises receiving a write request that would cause the JSON document in the compressed format to exceed the capacity of the table row;
said deciding to compact the JSON document is in response to said receiving the write request that would cause the JSON document in the compressed format to exceed the capacity of the table row.
3. The method of claim 2 further comprising in response to said compacting the JSON document in the compressed format, retaining the JSON document in the compressed format in the table row.
4. The method of claim 1 wherein:
said storing the JSON document in the compressed format in persistent storage comprises persisting the JSON document in the compressed format outside of a table row that is in persistent storage;
the method further comprises in response to said compacting the JSON document in the compressed format, inlining the JSON document in the compressed format in the table row.
5. The method of claim 1 wherein said compacting the JSON document in the compressed format consists essentially of one selected from the group consisting of:
a partial rewrite of the JSON document in the compressed format in persistent storage, and
an entire rewrite of the JSON document in the compressed format in persistent storage.
6. The method of claim 1 wherein said automatically deciding to compact the JSON document is caused by one selected from the group consisting of:
database archiving,
a cache write back,
a commit of a database transaction,
a numeric compaction threshold being crossed,
an access to the JSON document in the compressed format, and
moving the JSON document in the compressed format from a faster storage tier to a slower storage tier.
7. The method of claim 6 wherein the numeric compaction threshold is selected from the group consisting of:
a count of free disk space,
a count of requests to modify the JSON document that were fulfilled without entirely rewriting the JSON document in the compressed format in persistent storage,
a count of bytes or storage blocks appended onto the JSON document in the compressed format without entirely rewriting the JSON document in the compressed format,
a count of requests to modify the JSON document that increased the size of the JSON document in the compressed format without entirely rewriting persistent JSON document,
a ratio or difference between the last compact size of the JSON document in the compressed format and the accumulated additional size of the JSON document in the compressed format, and
a duration elapsed, without entirely rewriting the JSON document in the compressed format in persistent storage, since the size of the JSON document in the compressed format in persistent storage was last increased.
8. The method of claim 6 wherein said automatically deciding to compact the JSON document is based on at least one of selected from the group consisting of:
a timer,
multiple numeric compaction thresholds,
a threshold that specifies a count of numeric compaction thresholds, and
a forecast of future accesses to the JSON document in the compressed format.
9. The method of claim 1 wherein:
an activity occurs in one selected from the group consisting of the foreground and the background;
the activity is one selected from the group consisting of:
said deciding to compact the JSON document, and
said compacting the JSON document in the compressed format.
10. The method of claim 1 wherein said automatically deciding to compact the JSON document is not in response to a client request.
11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:
storing a JavaScript object notation (JSON) document in a compressed format in persistent storage;
automatically deciding, after said storing the JSON document in the compressed format in persistent storage, to compact the JSON document in the compressed format in persistent storage;
compacting, in response to said automatically deciding to compact the JSON document in the compressed format in persistent storage, the JSON document in the compressed format in persistent storage.
12. The one or more non-transitory computer-readable media of claim 11 wherein:
said storing the JSON document in the compressed format in persistent storage comprises inlining the JSON document in the compressed format in a table row in persistent storage;
the instructions further cause receiving a write request that would cause the JSON document in the compressed format to exceed the capacity of the table row;
said deciding to compact the JSON document is in response to said receiving the write request that would cause the JSON document in the compressed format to exceed the capacity of the table row.
13. The one or more non-transitory computer-readable media of claim 12 wherein the instructions further cause in response to said compacting the JSON document in the compressed format, retaining the JSON document in the compressed format in the table row.
14. The one or more non-transitory computer-readable media of claim 11 wherein:
said storing the JSON document in the compressed format in persistent storage comprises persisting the JSON document in the compressed format outside of a table row that is in persistent storage;
the instructions further cause in response to said compacting the JSON document in the compressed format, inlining the JSON document in the compressed format in the table row.
15. The one or more non-transitory computer-readable media of claim 11 wherein said compacting the JSON document in the compressed format consists essentially of one selected from the group consisting of:
a partial rewrite of the JSON document in the compressed format in persistent storage, and
an entire rewrite of the JSON document in the compressed format in persistent storage.
16. The one or more non-transitory computer-readable media of claim 11 wherein said automatically deciding to compact the JSON document is caused by one selected from the group consisting of:
database archiving,
a cache write back,
a commit of a database transaction,
a numeric compaction threshold being crossed,
an access to the JSON document in the compressed format, and
moving the JSON document in the compressed format from a faster storage tier to a slower storage tier.
17. The one or more non-transitory computer-readable media of claim 16 wherein the numeric compaction threshold is selected from the group consisting of:
a count of free disk space,
a count of requests to modify the JSON document that were fulfilled without entirely rewriting the JSON document in the compressed format in persistent storage,
a count of bytes or storage blocks appended onto the JSON document in the compressed format without entirely rewriting the JSON document in the compressed format,
a count of requests to modify the JSON document that increased the size of the JSON document in the compressed format without entirely rewriting persistent JSON document,
a ratio or difference between the last compact size of the JSON document in the compressed format and the accumulated additional size of the JSON document in the compressed format, and
a duration elapsed, without entirely rewriting the JSON document in the compressed format in persistent storage, since the size of the JSON document in the compressed format in persistent storage was last increased.
18. The one or more non-transitory computer-readable media of claim 16 wherein said automatically deciding to compact the JSON document is based on at least one of selected from the group consisting of:
a timer,
multiple numeric compaction thresholds,
a threshold that specifies a count of numeric compaction thresholds, and
a forecast of future accesses to the JSON document in the compressed format.
19. The one or more non-transitory computer-readable media of claim 11 wherein:
an activity occurs in one selected from the group consisting of the foreground and the background;
the activity is one selected from the group consisting of:
said deciding to compact the JSON document, and
said compacting the JSON document in the compressed format.
20. The one or more non-transitory computer-readable media of claim 11 wherein said automatically deciding to compact the JSON document is not in response to a client request.