Distributed tokenization using several substitution steps
A method for distributed tokenization of sensitive strings of characters, such as social security numbers, credit card numbers and the like, in a local server is disclosed. The method comprises the steps of receiving from a central server at least one, and preferably at least two, static token lookup tables, and receiving a sensitive string of characters. In a first tokenization step, a first substring of characters is substituted with a corresponding first token from the token lookup table(s) to form a first tokenized string of characters, wherein the first substring of characters is a substring of the sensitive string of characters. Thereafter, in a second step of tokenization, a second substring of characters is substituted with a corresponding second token from the token lookup table(s) to form a second tokenized string of characters, wherein the second substring of characters is a substring of the first tokenized string of characters. Optionally, one or more additional tokenization steps is/are used.
1. A method of tokenization, comprising:
receiving a string of characters in a first character domain at a local server;
accessing, by the local server, a first token table and a second token table, the first token table mapping each of a set of input values in the first character domain to a different token value comprising characters in the first character domain and characters in a second character domain different from the first character domain, the second token table mapping each of a set of input values in the first character domain to a different token value comprising characters in the second character domain;
tokenizing, by the local server, the accessed string of characters by replacing a first substring of the accessed string of characters with a first token value mapped to a value of the first substring by the first token table to form a first intermediate string of characters;
tokenizing, by the local server, a portion of the first intermediate string of characters comprising characters in the first character domain included in the first token value by replacing the portion of the first intermediate string of characters with a second token value mapped to a value of the portion of the first intermediate string of characters by the second token table to form a tokenized string of characters; and
transmitting, by the local server, the tokenized string of characters to a central server.
2. The method of claim 1 , further comprising modifying the first substring of the accessed string of characters before tokenizing the first substring of the accessed string of characters.
3. The method of claim 2 , wherein the first substring of the accessed string of characters is modified based at least in part on one or more initialization vectors.
4. The method of claim 1 , wherein the portion of the first intermediate string of characters comprises all characters of the first token value within the first character domain.
5. The method of claim 1 , wherein the tokenized string of characters comprises characters in the first character domain and characters in the second character domain.
6. The method of claim 1 , wherein the first character domain and the second character domain have no overlapping characters.
7. The method of claim 1 , wherein the first character domain comprises numeric characters and wherein the second character domain comprises alphabet characters.
8. A server, comprising:
a non-transitory computer-readable storage medium storing executable instructions that, when executed, perform steps comprising:
receiving a string of characters in a first character domain at the server;
accessing, by the server, a first token table and a second token table, the first token table mapping each of a set of input values in the first character domain to a different token value comprising characters in the first character domain and characters in a second character domain different from the first character domain, the second token table mapping each of a set of input values in the first character domain to a different token value comprising characters in the second character domain;
tokenizing, by the server, the accessed string of characters by replacing a first substring of the accessed string of characters with a first token value mapped to a value of the first substring by the first token table to form a first intermediate string of characters;
tokenizing, by the server, a portion of the first intermediate string of characters comprising characters in the first character domain included in the first token value by replacing the portion of the first intermediate string of characters with a second token value mapped to a value of the portion of the first intermediate string of characters by the second token table to form a tokenized string of characters; and
transmitting, by the server, the tokenized string of characters to a central server; and
a hardware processor configured to execute the instructions.
9. The server of claim 8 , further comprising modifying the first substring of the accessed string of characters before tokenizing the first substring of the accessed string of characters.
10. The server of claim 9 , wherein the first substring of the accessed string of characters is modified based at least in part on one or more initialization vectors.
11. The server of claim 8 , wherein the portion of the first intermediate string of characters comprises all characters of the first token value within the first character domain.
12. The server of claim 8 , wherein the tokenized string of characters comprises characters in the first character domain and characters in the second character domain.
13. The server of claim 8 , wherein the first character domain and the second character domain have no overlapping characters.
14. The server of claim 8 , wherein the first character domain comprises numeric characters and wherein the second character domain comprises alphabet characters.
15. A non-transitory computer-readable storage medium storing executable computer instructions that, when executed by a processor, cause the processor to perform steps comprising:
receiving a string of characters in a first character domain at a local server;
accessing, by the local server, a first token table and a second token table, the first token table mapping each of a set of input values in the first character domain to a different token value comprising characters in the first character domain and characters in a second character domain different from the first character domain, the second token table mapping each of a set of input values in the first character domain to a different token value comprising characters in the second character domain;
tokenizing, by the local server, the accessed string of characters by replacing a first substring of the accessed string of characters with a first token value mapped to a value of the first substring by the first token table to form a first intermediate string of characters;
tokenizing, by the local server, a portion of the first intermediate string of characters comprising characters in the first character domain included in the first token value by replacing the portion of the first intermediate string of characters with a second token value mapped to a value of the portion of the first intermediate string of characters by the second token table to form a tokenized string of characters; and
transmitting, by the local server, the tokenized string of characters to a central server.
16. The non-transitory computer-readable storage medium of claim 15 , further comprising modifying the first substring of the accessed string of characters before tokenizing the first substring of the accessed string of characters.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the first substring of the accessed string of characters is modified based at least in part on one or more initialization vectors.
18. The non-transitory computer-readable storage medium of claim 15 , wherein the portion of the first intermediate string of characters comprises all characters of the first token value within the first character domain.
19. The non-transitory computer-readable storage medium of claim 15 , wherein the tokenized string of characters comprises characters in the first character domain and characters in the second character domain.
20. The non-transitory computer-readable storage medium of claim 15 , wherein the first character domain and the second character domain have no overlapping characters.