Systems and methods for preparing email databases for analysis
The disclosed computer-implemented method for preparing email databases for analysis may include (1) identifying an email database that stores a plurality of emails in a plurality of tables that are formatted to be managed by a specific email application, (2) using a component of the specific email application to retrieve the plurality of emails from the database, (3) creating a denormalized dataset for the plurality of emails by combining email data from at least one table from the plurality of tables with email data from at least one other table from the plurality of tables, and (4) exporting at least a portion of the data from the denormalized dataset into at least one file in an interoperable format that is capable of being read by a plurality of applications. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for preparing email databases for analysis, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
identifying an email database that stores a plurality of emails in a plurality of tables that are formatted to be managed by a specific email application;
using a component of the specific email application to retrieve the plurality of emails from the database;
creating a denormalized dataset from the plurality of emails by combining email data from at least one table from the plurality of tables with email data from at least one other table from the plurality of tables, wherein the denormalized dataset has fewer tables than the email database;
determining a size of a data block to be read and analyzed by each application of a plurality of applications;
splitting at least a portion of the data from the denormalized dataset into a plurality of files, wherein a size of each file of the plurality of files is selected to be smaller than the size of the data block to be read and analyzed by each application of the plurality of applications; and
exporting the plurality of files in an interoperable format that is capable of being read and analyzed by the plurality of applications, wherein the plurality of applications analyze the plurality of files to determine at least one of a sentiment of the combined email data or a presence of malicious code in the combined email data.
2. The computer-implemented method of claim 1 , wherein the component of the specific email application comprises a dynamic link library that is used by the specific email application.
3. The computer-implemented method of claim 1 , wherein using the component of the specific email application to retrieve the plurality of emails comprises using an application programming interface to communicate with the component of the application.
4. The computer-implemented method of claim 1 , wherein the email database comprises a backup of a production email database where the plurality of emails originate.
5. The computer-implemented method of claim 1 , wherein creating the denormalized dataset comprises combining all of the tables within the plurality of tables into a single table.
6. The computer-implemented method of claim 1 , wherein the interoperable format comprises a columnar storage format.
7. A system for preparing email databases for analysis, the system comprising:
an identification module, stored in memory, that identifies an email database that stores a plurality of emails in a plurality of tables that are formatted to be managed by a specific email application;
a retrieval module, stored in memory, that uses a component of the specific email application to retrieve the plurality of emails from the database;
a creation module, stored in memory, that creates a denormalized dataset from the plurality of emails by combining email data from at least one table from the plurality of tables with email data from at least one other table from the plurality of tables, wherein the denormalized dataset has fewer tables than the email database;
an export module, stored in memory, that determines a size of a data block to be read and analyzed by each application of a plurality of applications, that splits at least a portion of the data from the denormalized dataset into a plurality of files, wherein a size of each file of the plurality of files is selected to be smaller than the size of the data block to be read and analyzed by each application of the plurality of applications, and that exports the plurality of files in an interoperable format that is capable of being read and analyzed by the plurality of applications, wherein the plurality of applications analyze the plurality of files to determine at least one of a sentiment of the combined email data or a presence of malicious code in the combined email data; and
at least one physical processor configured to execute the identification module, the retrieval module, the creation module, and the export module.
8. The system of claim 7 , wherein the component of the specific email application comprises a dynamic link library that is used by the specific email application.
9. The system of claim 7 , wherein the retrieval module uses the component of the specific email application to retrieve the plurality of emails by using an application programming interface to communicate with the component of the application.
10. The system of claim 7 , wherein the email database comprises a backup of a production email database where the plurality of emails originate.
11. The system of claim 7 , wherein the creation module creates the denormalized dataset by combining all of the tables within the plurality of tables into a single table.
12. The system of claim 7 , wherein the interoperable format comprises a columnar storage format.
13. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
identify an email database that stores a plurality of emails in a plurality of tables that are formatted to be managed by a specific email application;
use a component of the specific email application to retrieve the plurality of emails from the database;
create a denormalized dataset for the plurality of emails by combining email data from at least one table from the plurality of tables with email data from at least one other table from the plurality of tables, wherein the denormalized dataset has fewer tables than the email database;
determine a size of a data block to be read and analyzed by each application of a plurality of applications;
split at least a portion of the data from the denormalized dataset into a plurality of files, wherein a size of each file of the plurality of files is selected to be smaller than the size of the data block to be read and analyzed by each application of the plurality of applications; and
export the plurality of files in an interoperable format that is capable of being read and analyzed by the plurality of applications, wherein the plurality of applications analyze the plurality of files to determine at least one of a sentiment of the combined email data or a presence of malicious code in the combined email data.
14. The non-transitory computer-readable medium of claim 13 , wherein the component of the specific email application comprises a dynamic link library that is used by the specific email application.
15. The non-transitory computer-readable medium of claim 13 , wherein the one or more computer-readable instructions cause the computing device to use the component of the specific email application to retrieve the plurality of emails by using an application programming interface to communicate with the component of the application.
16. The non-transitory computer-readable medium of claim 13 , wherein the email database comprises a backup of a production email database where the plurality of emails originate.
17. The non-transitory computer-readable medium of claim 13 , wherein the one or more computer-readable instructions cause the computing device to create the denormalized dataset by combining all of the tables within the plurality of tables into a single table.