Restoring non-transactional messages in queues for advertisement data flow processing
View Patent ↗The disclosed online system includes a number of processing blocks, including a feeding service that sends a data stream and a consuming service that receives a data stream, without sending back acknowledgements for the data (i.e., “nontransactional”). The system handles failure in the feeding service and/or the consuming service by adding reference points into the data stream and backing up a current sum that is maintained by the consuming service. Upon a failure of the consuming service, the system obtains the last reference point, restores the backup copy of the database, and starts reading the backup copy from that check point. To address a failure of the feeding service, the feeding service creates an identifier for each message in the data stream, and upon failure, the consuming service checks for messages that have the same identifier within a specified period of time, and discards any duplicate messages.
1. A method comprising:
receiving, by a first processing service, a data stream from an ad server, the data stream comprising a first set of messages and at least one reference point, each message from the first set of messages formatted as an ordered set including an ad identifier and cost information about a served ad;
for each message in the first set of messages, identifying a cost limit identifier associated with the ad identifier;
generating, by the first processing service, a second set of messages, each message from the second set of messages formatted as an ordered set including the cost limit identifier and the cost information about the served ad;
storing, by a second processing service, the second set of messages in a database at specified interval;
responsive to detecting the at least one reference point, storing, by the second processing service, a copy of the database in a file system;
responsive to a failure of the second processing service, retrieving by the second processing service the at least one reference point;
retrieving the copy of the database from the file system; and
reading contents of the copy the database from a location corresponding to the at least one reference point, the copy of the data base retrieved from the file system.
2. The method of claim 1 , wherein the copy of the database includes at least a set of the first set of messages received up to an occurrence of the at least one reference point.
3. The method of claim 1 , wherein the file system is included in a computing system that is separate from another computing system executing the second processing service.
4. The method of claim 1 , further comprising storing, in another memory location separate from the file system, the at least one reference point.
5. The method of claim 1 , wherein the at least one reference point corresponds to a location in the data stream where a process can be restarted and resumed in the event of a failure of a processing service.
6. The method of claim 1 , wherein for each message in the second set of messages, incrementing a counter value with the cost information about the served ad.
7. A method comprising:
receiving, by a first processing service, a data stream from an ad server, the data stream comprising a first set of messages and at least one reference point, each message of the first set of messages formatted as an ordered set including an ad identifier and cost information about a served ad;
for each message in the first set of messages:
generating a hash value based on components of the message,
generating a message identifier based at least in part on the hash value, and
identifying a cost limit identifier associated with the ad identifier;
generating, by the first processing service, a second set of messages, each message from the second set of messages including a message identifier and formatted as an ordered set including the cost limit identifier and the cost information about a served ad;
receiving, by a second processing service, the second set of messages;
determining, within a specified period, whether at least two messages from the second set of messages have a same message identifier; and
discarding at least one of the two messages determined to have the same message identifier.
8. The method of claim 7 , wherein the second set of messages includes at least two reference points, wherein a reference point corresponds to location in the data stream where a process can be restarted in the event of a failure of a processing service.
9. The method of claim 7 , wherein the specified period corresponds to a time period between two consecutive reference points in the data stream.
10. The method of claim 7 , wherein generating a message identifier based at least in part on the hash value further comprises adding a plurality of bits of entropy to the hashed value.
11. A non-transitory computer readable medium comprising instructions that when executed by a processor cause the processor to:
receive, by a first processing service, a data stream from an ad server, the data stream comprising a first set of messages and at least one reference point, each message from the first set of messages formatted as an ordered set including an ad identifier and cost information about a served ad;
for each message in the first set of messages, identify a cost limit identifier associated with the ad identifier;
generate, by the first processing service, a second set of messages, each message from the second set of messages formatted as an ordered set including the cost limit identifier and the cost information about the served ad;
store, by a second processing service, the second set of messages in a database at specified interval;
responsive to a detected at least one reference point, store, by the second processing service, a copy of the database in a file system;
responsive to a failure of the second processing service, retrieving by the second processing service the at least one reference point;
retrieve the copy of the database from the file system; and
read contents of the copy the database from a location corresponding to the at least one reference point, the copy of the data base retrieved from the file system.
12. The non-transitory computer readable medium of claim 11 , wherein the copy of the database includes at least a set of the first set of messages received up to an occurrence of the at least one reference point.
13. The non-transitory computer readable medium of claim 11 , wherein the file system is included in a computing system that is separate from another computing system executing the second processing service.
14. The non-transitory computer readable medium of claim 11 , further comprising instructions that cause the processor to store, in another memory location separate from the file system, the at least one reference point.
15. The non-transitory computer readable medium of claim 11 , wherein the at least one reference point corresponds to a location in the data stream where a process can be restarted and resumed in the event of a failure of a processing service.
16. The non-transitory computer readable medium of claim 11 , further comprising instructions that cause the processor to:
for each message in the second set of messages, increment a counter value with the cost information about the served ad.
17. A non-transitory computer readable medium comprising instructions that when executed by a processor cause the processor to:
receive, by a first processing service, a data stream from an ad server, the data stream comprising a first set of messages and at least one reference point, each message of the first set of messages formatted as an ordered set including an ad identifier and cost information about a served ad;
for each message in the first set of messages:
generate a hash value based on components of the message,
generate a message identifier based at least in part on the hash value, and
identify a cost limit identifier associated with the ad identifier;
generate, by the first processing service, a second set of messages, each message from the second set of messages including a message identifier and formatted as an ordered set including the cost limit identifier and the cost information about a served ad;
receive, by a second processing service, the second set of messages;
determine, within a specified period, whether at least two messages from the second set of messages have a same message identifier; and
discard at least one of the two messages determined to have the same message identifier.
18. The non-transitory computer readable medium of claim 17 , wherein the second set of messages includes at least two reference points, wherein a reference point corresponds to location in the data stream where a process can be restarted in the event of a failure of a processing service.
19. The non-transitory computer readable medium of claim 17 , wherein the specified period corresponds to a time period between two consecutive reference points in the data stream.