Journal parsing for object event generation
A system can register a first client and a second client to respectively receive information about updates to a write-ahead log structured storage engine that comprises a log. The system can read an entry in the log, the entry being of an update type. The system can, in response to determining that the first client is registered to receive at least some of the information about updates that are of the update type, sending, to the first client, entry information about the entry. The system can, in response to determining that the second client is registered to receive information about updates that are of the update type, sending, to the second client, the entry information about the entry.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
registering a first client and a second client to respectively receive information about updates to a write-ahead log structured storage engine that comprises a log;
reading an entry in the log, the entry being of an update type, to enable centralized journal parsing by scaling up a number of clients while holding resource usage constant;
in response to determining that the first client is registered to receive at least some of the information about updates that are of the update type, sending, to the first client, entry information about the entry;
in response to determining that the second client is registered to receive the information about updates that are of the update type, sending, to the second client, the entry information about the entry; and
maintaining a first marker that identifies a first location that was last successfully-parsed for the first client in the log, wherein the first marker identifies a region major number within a chunk that stores information about the log.
2. The system of claim 1 , wherein the registering the first client to receive the information about updates comprises registering the first client to receive the at least some of the information about updates of the update type.
3. The system of claim 2 , wherein the update type is a first update type, and wherein the registering the first client to receive the at least some of the information about updates comprises:
registering the first client to receive the at least some of the information about a subset of possible update types.
4. The system of claim 1 , wherein the operations further comprise:
registering a third client to receive the information about updates to the write-ahead log structured storage engine; and
in response to determining that the third client is not registered to receive the at least some of the information about updates that are of the update type, determining to refrain from the sending of the entry information to the second client.
5. The system of claim 1 , wherein the operations further comprise:
maintaining a second marker that identifies a second location that was last successfully-parsed for the second client in the log.
6. The system of claim 1 , wherein the first marker further identifies a region minor number and an offset of the first marker within the chunk that stores information about the log.
7. The system of claim 1 , wherein the operations further comprise:
maintaining the first marker that identifies the first location that was last successfully-parsed for the first client in a first partition of the log; and
maintaining a second marker that identifies a second location that was last successfully-parsed for the first client in a second partition of the log.
8. A method, comprising:
registering, by a system comprising a processor, a first client and a second client to respectively receive information about updates to a write-ahead log structured storage engine that comprises a log;
reading, by the system, an entry in the log, the entry being of an update type, to enable centralized journal parsing by scaling up a number of clients while maintaining an amount of resource usage;
sending, by the system, entry information about the entry, to the first client, in response to determining that the first client is registered to receive at least some of the information about updates that are of the update type;
sending, by the system, the entry information about the entry, to the second client, in response to determining that the second client is registered to receive the information about updates that are of the update type; and
maintaining, by the system, a first marker that identifies a first location that was last successfully-parsed for the first client in a first partition of the log.
9. The method of claim 8 , wherein the registering the first client to receive the information about updates comprises registering the first client to receive the at least some of the information about updates of the update type.
10. The method of claim 9 , wherein the update type is a first update type, and wherein the registering the first client to receive the at least some of the information about updates comprises:
registering, by the system, the first client to receive the at least some of the information about a subset of possible update types.
11. The method of claim 8 , further comprising:
registering, by the system, a third client to receive the information about updates to the write-ahead log structured storage engine; and
determining, by the system, to refrain from the sending of the entry information to the second client, in response to determining that the third client is not registered to receive the at least some of the information about updates that are of the update type.
12. The method of claim 8 , further comprising:
maintaining, by the system, the first marker that identifies the first location that was last successfully-parsed for the first client in the log; and
maintaining, by the system, a second marker that identifies a second location that was last successfully-parsed for the second client in the log.
13. The method of claim 8 , wherein the first marker identifies a region major number, a region minor number, and an offset of the first marker within a chunk that stores information about the log.
14. The method of claim 8 , further comprising:
maintaining, by the system, a second marker that identifies a second location that was last successfully-parsed for the first client in a second partition of the log.
15. A non-transitory computer-readable medium comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising:
registering a first client, a second client, and a third client to respectively receive information about updates to a write-ahead log structured storage engine that comprises a log;
reading an entry in the log, the entry being of an update type, to enable centralized journal parsing by scaling up a number of clients without modifying resource usage;
in response to determining that the first client is registered to receive at least some of the information about updates that are of the update type, sending, to the first client, entry information about the entry;
in response to determining that the second client is registered to receive the information about updates that are of the update type, sending, to the second client, the entry information about the entry; and
Maintaining a first marker that identifies a first location that was last successfully-parsed for the first client in a first partition of the log, wherein the first marker identifies a region major number within a chunk that stores information about the log.
16. The non-transitory computer-readable medium of claim 15 , wherein the registering the first client to receive the information about updates comprises registering the first client to receive the at least some of the information about updates of the update type.
17. The non-transitory computer-readable medium of claim 16 , wherein the update type is a first update type, and wherein the registering the first client to receive the at least some of the information about updates comprises:
registering the first client to receive the at least some of the information about a subset of possible update types.
18. The non-transitory computer-readable medium of claim 15 , wherein the operations further comprise:
in response to determining that the third client is not registered to receive the at least some of the information about updates that are of the update type, determining to refrain from the sending of the entry information to the second client.
19. The non-transitory computer-readable medium of claim 15 , wherein the operations further comprise:
maintaining a first marker that identifies a first location that was last successfully-parsed for the first client in the log; and
maintaining a second marker that identifies a second location that was last successfully-parsed for the second client in the log.
20. The non-transitory computer-readable medium of claim 19 , wherein the first marker further identifies a region minor number and an offset of the first marker within the chunk that stores information about the log.