Sharing events and other metrics in native applications
Disclosed is an execution information sharing system that writes execution information to a provider target (and other targets) in a secure manner. Execution information generated by an application may be written to a consumer stage, wherein the application is shared by a provider account of a data exchange with a consumer account that executes the application. A consumer exchange service (ES) of the data exchange may send a request to a copy service of the data exchange to copy the execution information from the consumer stage to the provider stage, wherein the consumer ES is a part of the data exchange and is protected from actions of the consumer account. A copy operation may be executed to copy the execution information from the consumer stage to the provider stage using the copy service of the data exchange. The execution information is ingested from the provider stage to a provider table.
1. A method comprising:
writing execution information generated by an application to a consumer stage, wherein the application is shared by a provider account of a data exchange with a consumer account that executes the application;
sending, by a consumer exchange service (ES) of the data exchange, a request to a copy service of the data exchange to copy the execution information from the consumer stage to the provider stage, wherein the consumer ES is a part of the data exchange and is protected from actions of the consumer account;
executing, by a processing device, a copy operation to copy the execution information from the consumer stage to the provider stage using the copy service of the data exchange; and
ingesting the execution information from the provider stage to a provider table using an ingestion service of the data exchange.
2. The method of claim 1 , further comprising:
sending to the consumer ES, a notification that the execution information is ready to be copied from the consumer stage to a provider stage, wherein the notification comprises source and destination information for executing the copy operation; and
determining, based on the source and destination information, an encryption key of the consumer account and an encryption key of the provider account, wherein the consumer ES includes the source and destination information, the encryption key of the consumer account and the encryption key of the provider account as part of the request to the copy service.
3. The method of claim 2 , further comprising:
decrypting the execution information using the encryption key of the consumer account; and
encrypting the execution information using the encryption key of the provider account.
4. The method of claim 2 , wherein:
the source information comprises a source file name of the execution information, a source volume ID of the consumer account, and an encryption key ID of the consumer account; and
the destination information comprises a destination file name for the execution information, a destination volume ID of the provider account, and an encryption key ID of the provider account.
5. The method of claim 1 , wherein a callback of the copy operation triggers the ingestion of the execution information to the provider table.
6. The method of claim 5 , wherein the consumer account is located on a first shard of the data exchange and the provider account is located on a second shard of the data exchange, and wherein the callback of the copy operation sends a global message to the ingestion service on the second shard to trigger the ingestion of the execution information to the provider table.
7. The method of claim 5 , wherein the consumer account is located on a first region of the data exchange and the provider account is located on a second region of the data exchange.
8. The method of claim 7 , wherein executing the copy operation comprises:
performing a first copy operation of the execution information from the consumer stage to a cross-region stage, wherein a callback of the first copy operation sends a global message to the copy service on the second region of the data exchange; and
in response to receiving the global message, performing a second copy operation of the execution information from the cross-region stage to the provider stage, wherein a callback of the second copy operation triggers ingestion of the execution information from the provider stage to the provider table.
9. A system comprising:
a memory; and
a processing device operatively coupled to the memory, the processing device to:
write execution information generated by an application to a consumer stage, wherein the application is shared by a provider account of a data exchange with a consumer account that executes the application;
send, by a consumer ES of the data exchange, a request to a copy service of the data exchange to copy the execution information from the consumer stage to the provider stage, wherein the consumer ES is a part of the data exchange and is protected from actions of the consumer account;
execute a copy operation to copy the execution information from the consumer stage to the provider stage using the copy service of the data exchange; and
ingest the execution information from the provider stage to a provider table using an ingestion service of the data exchange.
10. The system of claim 9 , wherein the processing device is further to:
send to the consumer ES, a notification that the execution information is ready to be copied from the consumer stage to a provider stage, wherein the notification comprises source and destination information for executing the copy operation; and
determine, based on the source and destination information, an encryption key of the consumer account and an encryption key of the provider account, wherein the consumer ES includes the source and destination information, the encryption key of the consumer account and the encryption key of the provider account as part of the request to the copy service.
11. The system of claim 10 , wherein the processing device is further to:
decrypt the execution information using the encryption key of the consumer account; and
encrypt the execution information using the encryption key of the provider account.
12. The system of claim 10 , wherein:
the source information comprises a source file name of the execution information, a source volume ID of the consumer account, and an encryption key ID of the consumer account; and
the destination information comprises a destination file name for the execution information, a destination volume ID of the provider account, and an encryption key ID of the provider account.
13. The system of claim 9 , wherein a callback of the copy operation causes the processing device to trigger the ingestion of the execution information to the provider table.
14. The system of claim 13 , wherein the consumer account is located on a first shard of the data exchange and the provider account is located on a second shard of the data exchange, and wherein the callback of the copy operation causes the processing device to send a global message to the ingestion service on the second shard to trigger the ingestion of the execution information to the provider table.
15. The system of claim 13 , wherein the consumer account is located on a first region of the data exchange and the provider account is located on a second region of the data exchange.
16. The system of claim 15 , wherein to execute the copy operation, the processing device is to:
perform a first copy operation of the execution information from the consumer stage to a cross-region stage, wherein a callback of the first copy operation sends a global message to the copy service on the second region of the data exchange; and
in response to receiving the global message, perform a second copy operation of the execution information from the cross-region stage to the provider stage, wherein a callback of the second copy operation triggers ingestion of the execution information from the provider stage to the provider table.
17. A non-transitory computer-readable medium having instructions stored thereon which, when executed by a processing device cause the processing device to:
write execution information generated by an application to a consumer stage, wherein the application is shared by a provider account of a data exchange with a consumer account that executes the application;
send, by a consumer ES of the data exchange, a request to a copy service of the data exchange to copy the execution information from the consumer stage to the provider stage, wherein the consumer ES is a part of the data exchange and is protected from actions of the consumer account;
execute, by the processing device, a copy operation to copy the execution information from the consumer stage to the provider stage using the copy service of the data exchange; and
ingest the execution information from the provider stage to a provider table using an ingestion service of the data exchange.
18. The non-transitory computer-readable medium of claim 17 , wherein the processing device is further to:
send to the consumer ES, a notification that the execution information is ready to be copied from the consumer stage to a provider stage, wherein the notification comprises source and destination information for executing the copy operation; and
determine, based on the source and destination information, an encryption key of the consumer account and an encryption key of the provider account, wherein the consumer ES includes the source and destination information, the encryption key of the consumer account and the encryption key of the provider account as part of the request to the copy service.
19. The non-transitory computer-readable medium of claim 18 , wherein the processing device is further to:
decrypt the execution information using the encryption key of the consumer account; and
encrypt the execution information using the encryption key of the provider account.
20. The non-transitory computer-readable medium of claim 18 , wherein:
the source information comprises a source file name of the execution information, a source volume ID of the consumer account, and an encryption key ID of the consumer account; and
the destination information comprises a destination file name for the execution information, a destination volume ID of the provider account, and an encryption key ID of the provider account.
21. The non-transitory computer-readable medium of claim 17 , wherein a callback of the copy operation causes the processing device to trigger the ingestion of the execution information to the provider table.
22. The non-transitory computer-readable medium of claim 21 , wherein the consumer account is located on a first shard of the data exchange and the provider account is located on a second shard of the data exchange, and wherein the callback of the copy operation causes the processing device to send a global message to the ingestion service on the second shard to trigger the ingestion of the execution information to the provider table.
23. The non-transitory computer-readable medium of claim 21 , wherein the consumer account is located on a first region of the data exchange and the provider account is located on a second region of the data exchange.
24. The non-transitory computer-readable medium of claim 23 , wherein to execute the copy operation, the processing device is to:
perform a first copy operation of the execution information from the consumer stage to a cross-region stage, wherein a callback of the first copy operation sends a global message to the copy service on the second region of the data exchange; and
in response to receiving the global message, perform a second copy operation of the execution information from the cross-region stage to the provider stage, wherein a callback of the second copy operation triggers ingestion of the execution information from the provider stage to the provider table.