Information security apparatus and methods for credential dump authenticity verification
In some embodiments, an apparatus includes a memory, storing processor-executable instructions, blacklist terms, and credential dump records, and a processor. The processor receives repository data from targeted remote repositories and stores the repository data as a potential credential dump in the memory when the repository data includes a credential dump attribute. The processor stores the potential credential dump as a probable credential dump when the potential credential dump does not include a blacklist term, in which case the processor also detects a format and delimiter of the probable credential dump. Based on the format and delimiter, pairs of usernames and associated passwords are identified and hashed. If a percentage of the hashes not associated with the credential dump records exceeds a predetermined threshold, the probable credential dump is deemed authentic.
1. An apparatus, comprising:
a memory storing processor-executable instructions, a plurality of blacklist terms previously-identified as included in an inauthentic credential dump, and a plurality of credential dump records, each credential dump record from the plurality of credential dump records including an associated plurality of hashes; and
at least one processor, operably coupled to the memory and configured to execute the processor-executable instructions to:
receive repository data from a plurality of targeted remote repositories; determine the repository data omits each blacklist term from the plurality of blacklist terms;
in response to the determination that the repository data omits each blacklist term from the plurality of blacklist terms:
detect a common format and a common delimiter of the repository data;
identify a plurality of pairs of usernames and associated passwords of the repository data based on the common format and the common delimiter;
generate a hash for each pair of usernames and associated passwords from the plurality of pairs of usernames and associated passwords to produce a plurality of hashes;
compare the plurality of hashes to the plurality of hashes associated with the plurality of credential dump records stored in the memory to determine a percentage of the plurality of hashes that are not associated with the plurality of credential dump records;
identify the repository data as an authentic credential dump in response to the determination that the percentage is larger than a predetermined threshold; and
send a signal identifying an intrusion into a computer system associated with the repository data after the repository data is identified as an authentic credential clump; and
wherein the repository data is received from a first targeted remote repository of the plurality of targeted remote repositories, periodically, at a first rate that is a function of the first targeted remote repository, and the repository data is received from a second targeted remote repository of the plurality of targeted remote repositories, periodically, at a second rate that is a function of the second targeted remote repository.
2. The apparatus of claim 1 , wherein the repository data is received from the plurality of targeted remote repositories, and the instruction to receive is performed repeatedly and at a predetermined rate.
3. The apparatus of claim 1 , wherein the repository data is received from the plurality of targeted remote repositories when a change is detected at a targeted remote repository or the plurality of targeted remote repositories.
4. The apparatus of claim 1 , wherein detecting the common delimiter of the repository data includes identifying a predetermined number of consecutive lines of the repository data that each include a common delimiter type, the detecting includes detecting the common delimiter when the predetermined number exceeds a threshold.
5. The apparatus of claim 1 , wherein detecting the common format and the common delimiter of the repository data includes identifying a predetermined number of consecutive lines of the repository data in which respective usernames of the consecutive lines of the repository data are indexed at a common index position, and the detecting includes detecting the common format and the common delimiter when the predetermined number exceeds a threshold.
6. The apparatus of claim 1 , wherein each pair of usernames and associated passwords from the plurality of pairs of usernames and associated passwords includes the associated username concatenated with the associated password.
7. The apparatus of claim 1 , wherein each username of the plurality of pairs of usernames and associated passwords is an email address.
8. The apparatus of claim 1 , wherein receiving repository data from the plurality of targeted remote repositories is performed using web scraping.
9. A method, comprising:
receiving, using a processor, remote source data from a plurality of targeted remote sources;
determining the remote source data omits each blacklist term from the plurality of blacklist terms;
in response to the determination that the remote source data omits each blacklist term from the plurality of blacklist terms:
storing a plurality of credential pairs of the remote source data, in a memory that is operably coupled to the processor;
detecting a format of the remote source data including identifying a plurality of usernames and the plurality of passwords;
normalizing, using the processor, the plurality of credential pairs into a concatenated, delimiter-free format, the normalizing being based on the plurality of usernames and the plurality of passwords;
converting, using the processor, the normalized plurality of credential pairs into a plurality of hashes,
comparing, using the processor, the plurality of hashes to previously-collected credential dump data to determine a percentage of the plurality of hashes that are not included in the previously-collected credential dump data;
identifying, using the processor, the remote source data as including an authentic credential dump in response to the determination that the percentage of the plurality of hashes that are not included in the previously-collected credential dump data, is larger than a predetermined threshold; and
sending a signal identifying an intrusion into a computer system associated with the remote source data after the remote source data is identified as including an authentic credential dump; and
wherein the receiving the remote source data includes receiving the remote source data from a first targeted remote source of the plurality of targeted remote sources, periodically, at a first rate that is a function of the first targeted remote source, and from a second targeted remote source, of the plurality of targeted remote sources periodically, at a second rate that is a function of the second targeted remote source.
10. The method of claim 9 , wherein the detecting includes detecting a delimiter that recurs on a consecutive plurality of lines of the remote source data, the normalizing being based on the delimiter.
11. The method of claim 10 , wherein:
each username of the plurality of usernames is disposed in the remote source data before a delimiter of the detected recurring delimiters; and
each password of the plurality of passwords is disposed in the remote source data after a delimiter of the detected recurring delimiters.
12. The method of claim 10 , wherein:
each username of the plurality of usernames is disposed in the remote source data after a delimiter of the detected recurring delimiters; and
each password of the plurality of passwords is disposed in the remote source data before a delimiter of the detected recurring delimiters.
13. The method of claim 10 , wherein each blacklist term from the plurality of blacklist terms being previously-identified as included in an inauthentic credential dump.
14. The method of claim 10 , wherein the receiving the remote source data is performed repeatedly and at a predetermined rate.
15. A method, comprising:
storing a plurality of blacklist terms previously-identified as included in an inauthentic credential dump, and a plurality of credential dump records;
receiving, using a processor, remote source data from a plurality of targeted remote sources;
determining the remote source data omits each blacklist term from the plurality of blacklist terms;
in response to the determination that the remote source data omits each blacklist term from the plurality of blacklist terms:
storing, in a memory that is operably coupled to the processor, a plurality of credential pairs of the remote source data, each credential pair of the plurality of credential pairs including an associated username and an associated password;
comparing, using the processor, the plurality of credential pairs to previously-collected credential dump data to determine a percentage of the plurality of credential pairs that are not included in the previously-collected credential dump data;
identifying, using the processor, the remote source data as including an authentic credential dump in response to the determination that the percentage of the plurality of credential pans that are not included in the previously-collected credential dump data is larger than a predetermined threshold; and
sending a signal identifying an intrusion into a computer system associated with the remote source data after identifying the remote source data as including an authentic credential dump; and
wherein the receiving the remote source data includes receiving the remote source data from a first targeted remote source of the plurality of targeted remote sources, periodically, at a first rate that is a function of the first targeted remote source, and from a second targeted remote source of the plurality of targeted remote sources, periodically, at a second rate that is a function of the second targeted remote source.
16. The method of claim 15 , wherein the receiving the remote source data is performed repeatedly and at a predetermined rate.
17. The method of claim 15 , wherein the receiving the remote source data is performed using web scraping.