Systems and methods for updating email analytics databases
The disclosed computer-implemented method for updating email analytics databases may include (1) identifying an email database with a native format and an email analytics database that stores a copy of data in the email database in an analytics-friendly format that is denormalized relative to the native format of the email database, (2) capturing a log file comprising information that is about at least one recent change to the email database and that is formatted using the native format of the email database, (3) extracting the information about the recent change to the email database from the log file by transforming the information from the native format of the email database into the analytics-friendly format, and (4) updating the email analytics database to reflect the recent change to the email database by using the extracted information. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for updating email analytics databases, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
identifying an email database with a native format and an email analytics database that stores a copy of at least a portion of the data in the email database in an analytics-friendly format that is denormalized relative to the native format of the email database;
capturing log files each comprising information about at least one recent change to the email database and is formatted using the native format of the email database by monitoring the email database in order to determine when a new log file is created and capturing the new log file as soon as the new log file is closed to provide to the email analytics database up-to-date information about the at least one recent change;
extracting the information about the at least one recent change to the email database from the captured log files by transforming the information from the native format of the email database into the analytics-friendly format of the email analytics database;
locating the information about the at least one recent change based on the analytics-friendly format of the log files;
updating the email analytics database with a real-time stream of data reflecting the at least one recent change in the captured log files to the email database by using the extracted information that is in the analytics-friendly format; and
sending the real-time stream of data reflecting the at least one recent change in the captured log files to a stream analysis tool for analysis in real time as updates occur in the email analytics database using the analytics-friendly format.
2. The computer-implemented method of claim 1 , wherein the analytics-friendly format is denormalized relative to the native format of the email database due to being composed of fewer tables than the native format.
3. The computer-implemented method of claim 1 , wherein the analytics-friendly format is readable by at least one application that is not capable of reading the native format.
4. The computer-implemented method of claim 1 , wherein the email database comprises at least one of:
a backup copy of a production email database where a plurality of emails originate; or
a production email database where a plurality of emails originate.
5. The computer-implemented method of claim 1 , wherein the stream analysis tool comprises at least one of:
a SPARK stream analysis tool; or
a STORM stream analysis tool.
6. The computer-implemented method of claim 1 , wherein the analytics-friendly format is denormalized relative to the native format of the email database due to being composed of a single table.
7. The computer-implemented method of claim 1 , wherein the analytics-friendly format combines data from multiple tables from the email database.
8. A system for updating email analytics databases, the system comprising:
an identification module, stored in memory, that identifies an email database with a native format and an email analytics database that stores a copy of at least a portion of the data in the email database in an analytics-friendly format that is denormalized relative to the native format of the email database;
a capturing module, stored in memory, that captures log files each comprising information that is about at least one recent change to the email database and that is formatted using the native format of the email database by monitoring the email database in order to determine when a new log file is created and capturing the new log file as soon as the new log file is closed to provide to the email analytics database up-to-date information about the at least one recent change;
an extraction module, stored in memory, that extracts the information about the at least one recent change to the email database from the captured log files by transforming the information from the native format of the email database into the analytics-friendly format of the email analytics database and locates the information about the at least one recent change based on the analytics-friendly format of the log files;
an updating module, stored in memory, that updates the email analytics database with a real-time stream of data reflecting the at least one recent change in the captured log files to the email database by using the extracted information that is in the analytics-friendly format;
a sending module, stored in memory, that sends the real-time stream of data reflecting the at least one recent change in the captured log files to a stream analysis tool for analysis in real time as updates occur in the email analytics database using the analytics-friendly format; and
at least one physical processor configured to execute the identification module, the capturing module, the extraction module, and the updating module.
9. The system of claim 8 , wherein the analytics-friendly format is denormalized relative to the native format of the email database due to being composed of fewer tables than the native format.
10. The system of claim 8 , wherein the analytics-friendly format is readable by at least one application that is not capable of reading the native format.
11. The system of claim 8 , wherein the email database comprises a backup copy of a production email database where a plurality of emails originate.
12. The system of claim 8 , wherein the email database comprises a production email database where a plurality of emails originate.
13. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
identify an email database with a native format and an email analytics database that stores a copy of at least a portion of the data in the email database in an analytics-friendly format that is denormalized relative to the native format of the email database;
capture log files each comprising information that is about at least one recent change to the email database and that is formatted using the native format of the email database by monitoring the email database in order to determine when a new log file is created and capturing the new log file as soon as the new log file is closed to provide to the email analytics database up-to-date information about the at least one recent change;
extract the information about the at least one recent change to the email database from the captured log file by transforming the information from the native format of the email database into the analytics-friendly format of the email analytics database;
locate the information about the at least one recent change based on the analytics-friendly format of the log files;
update the email analytics database with a real-time stream of data reflecting the at least one recent change in the captured log files to the email database by using the extracted information that is in the analytics-friendly format; and
send the real-time stream of data reflecting the at least one recent change in the captured log files to a stream analysis tool for analysis in real time as updates occur in the email analytics database using the analytics-friendly format.
14. The non-transitory computer-readable medium of claim 13 , wherein the analytics-friendly format is denormalized relative to the native format of the email database due to being composed of fewer tables than the native format.
15. The non-transitory computer-readable medium of claim 13 , wherein the analytics-friendly format is readable by at least one application that is not capable of reading the native format.
16. The non-transitory computer-readable medium of claim 13 , wherein the email database comprises a backup copy of a production email database where a plurality of emails originate.