Communication data processing architecture
Various systems and methods are provided that allow reviewers to review and potentially flag processed communication data in order to spot potential malfeasance or illegal activity. Certain systems and methods includes modules for removing duplicate data from the communication data files, generating thread data for email data files, generating lexicon tag data for the data files, removing blacklisted data that meets blacklisted criteria, or generating group association data.
1. A computing system configured to display computer communication data, the computing system comprising:
one or more processors; and
a computer readable storage medium storing program instructions configured for execution by the one or more processors in order to cause the computing system to:
convert a plurality of communication data files comprising text into a pre-determined format to create a plurality of formatted communication data files;
generate offset tag data indicating a change in text position from an original position of text in an original data file of the plurality of communication data files to a formatted position of text in a formatted data file of the plurality of formatted communication data files, the change in text position being caused at least in part by the conversion of the plurality of communication data files into the pre-determined format; and
insert the offset tag data into a first subset of the plurality of formatted communication data files.
2. The computing system of claim 1 , wherein the stored program instructions further cause the one or more processors to process the plurality of formatted communication data files to generate a plurality of revised formatted communication data files by performing two or more of the following:
removing duplicate data among the plurality of formatted communication data files;
generating thread data for at least a second subset of the plurality of formatted communication data files;
removing, from the plurality of formatted communication data files, data that meets pre-identified blacklisted criteria; or
generating group association data for the plurality of formatted communication data files.
3. The computing system of claim 2 , wherein the stored program instructions further cause the one or more processors to:
store the revised formatted communication data files in persistent memory;
receive a keyword for searching;
access, from the persistent memory, the revised formatted communication data files for searching;
determine a character position to highlight based at least in part on the keyword and the offset tag data; and
transmit data for displaying content of a communication data file with a highlight at the character position.
4. The computing system of claim 1 , wherein the stored program instructions further cause the one or more processors to process the plurality of formatted communication data files to generate group association data based at least in part on people included in the plurality of formatted communication data files.
5. The computing system of claim 1 , wherein converting the plurality of communication data files includes applying a first conversion process to email data in the plurality of communication data files and applying a second conversion process to a different type of communication data in the plurality of communication data files, the first conversion process being different from the second conversion process.
6. The computing system of claim 1 , wherein:
converting the plurality of communication data files includes applying a first conversion process to a first communication data file in the plurality of communication data files and applying a second conversion process to second communication data file in the plurality of communication data files;
the first communication data file and the second communication data file are generated by different communication providers; and
the first conversion process is different from the second conversion process.
7. A computer-implemented method comprising:
converting a plurality of communication data files comprising text into a pre-determined format to create a plurality of formatted communication data files;
generate offset tag data indicating a change in text position from an original position of text in an original data file of the plurality of communication data files to a formatted position of text in a formatted data file of the plurality of formatted communication data files, the change in text position being caused at least in part by the conversion of the plurality of communication data files into the pre-determined format; and
insert the offset tag data into a subset of the plurality of formatted communication data files.
8. The computer-implemented method of claim 7 , further comprising processing the plurality of formatted communication data files to generate a plurality of revised formatted communication data files by performing two or more of the following:
removing duplicate data among the plurality of formatted communication data files;
generating thread data for at least a subset of the plurality of formatted communication data files;
removing, from the plurality of formatted communication data files, data that meets pre-identified blacklisted criteria; or
generating group association data for the plurality of formatted communication data files.
9. The computer-implemented method of claim 8 , further comprising:
storing the revised formatted communication data files in persistent memory;
receiving a keyword for searching;
accessing, from the persistent memory, the revised formatted communication data files for searching;
determining a character position to highlight based at least in part on the keyword and the offset tag data; and
transmitting data for displaying content of a communication data file with a highlight at the character position.
10. The computer-implemented method of claim 7 , further comprising processing the plurality of formatted communication data files to generate a plurality of revised formatted communication data files by generating group association data based at least in part on people included in the plurality of formatted communication data files.
11. The computer-implemented method of claim 10 , further comprising accessing human resources data to correlate each of the plurality of communication data files to one or more people.
12. The computer-implemented method of claim 7 , wherein converting the plurality of communication data files includes applying a first conversion process to email data in the plurality of communication data files and applying a second conversion process to a different type of communication data in the plurality of communication data files, the first conversion process being different from the second conversion process.
13. The computer-implemented method of claim 7 , wherein
converting the plurality of communication data files includes applying a first conversion process to a first communication data file in the plurality of communication data files and applying a second conversion process to second communication data file in the plurality of communication data files;
the first communication data file and the second communication data file are generated by different first communication providers; and
the first conversion process is different from the second conversion process.
14. A non-transitory computer-readable medium comprising one or more program instructions recorded thereon, the program instructions configured for execution by a computing system comprising one or more processors in order to cause the computing system to:
convert a plurality of communication data files comprising text into a pre-determined format to create a plurality of formatted communication data files;
generate offset tag data indicating a change in text position from an original position of text in an original data file of the plurality of communication data files to a formatted position of text in a formatted data file of the plurality of formatted communication data files, the change in text position being caused at least in part by the conversion of the plurality of communication data files into the pre-determined format; and
insert the offset tag data into a subset of the plurality of formatted communication data files.
15. The non-transitory computer-readable medium of claim 14 , wherein the one or more program instructions further cause the one or more processors to process the plurality of formatted communication data files to generate a plurality of revised formatted communication data files by performing two or more of the following:
removing duplicate data among the plurality of formatted communication data files;
generating thread data for at least a subset of the plurality of formatted communication data files;
removing, from the plurality of formatted communication data files, data that meets pre-identified blacklisted criteria; or
generating group association data for the plurality of formatted communication data files.
16. The non-transitory computer-readable medium of claim 15 , wherein the one or more program instructions further cause the one or more processors to:
store the revised formatted communication data files in persistent memory;
receive a keyword for searching;
access, from the persistent memory, the revised formatted communication data files for searching;
determining a character position to highlight based at least in part on the keyword and the offset tag data; and
transmitting data to display content of a communication data file with a highlight at the character position.
17. The non-transitory computer-readable medium of claim 14 , wherein the one or more program instructions further cause the one or more processors to process the plurality of formatted communication data files to generate a plurality of revised formatted communication data files by generating group association data based at least in part on people included in the plurality of formatted communication data files.
18. The non-transitory computer-readable medium of claim 17 , wherein the one or more program instructions further cause the one or more processors to access human resources data to correlate each of the plurality of communication data files to one or more people.
19. The non-transitory computer-readable medium of claim 14 , wherein:
converting the plurality of communication data files includes:
applying a first conversion process to email data in the plurality of communication data files; and
applying a second conversion process to a different type of communication data in the plurality of communication data files; and
the first conversion process is different from the second conversion process.
20. The non-transitory computer-readable medium of claim 14 , wherein:
converting the plurality of communication data files includes applying a first conversion process to a first communication data file in the plurality of communication data files and applying a second conversion process to a second communication data file in the plurality of communication data files;
the first communication data file and the second communication data file are generated by different first communication providers; and
the first conversion process is different from the second conversion process.