Distributed tokenization using several substitution steps
View Patent ↗A method for distributed tokenization of sensitive strings of characters, such as social security numbers, credit card numbers and the like, in a local server is disclosed. The method comprises the steps of receiving from a central server at least one, and preferably at least two, static token lookup tables, and receiving a sensitive string of characters. In a first tokenization step, a first substring of characters is substituted with a corresponding first token from the token lookup table(s) to form a first tokenized string of characters, wherein the first substring of characters is a substring of the sensitive string of characters. Thereafter, in a second step of tokenization, a second substring of characters is substituted with a corresponding second token from the token lookup table(s) to form a second tokenized string of characters, wherein the second substring of characters is a substring of the first tokenized string of characters. Optionally, one or more additional tokenization steps is/are used.
1. A method of tokenization, comprising:
accessing a string of characters;
accessing a first token table, a second token table, and a third token table, each of the first token table, the second token table, and the third token table mapping each of a set of input values to a different token value, wherein the first token table, the second token table and the third token table comprise different token tables;
tokenizing, by a processor, the accessed string of characters by replacing a first substring of the accessed string of characters with a first token value mapped to a value of the first substring by the first token table to form a first intermediate string of characters;
tokenizing, by the processor, the first intermediate string of characters by replacing a second substring of the first intermediate string of characters with a second token value mapped to a value of the second substring by the second token table to form a second intermediate string of characters; and
tokenizing, by the processor, the second intermediate string of characters by replacing a third substring of the second intermediate string of characters with a third token value mapped to a value of the third substring by the third token table to form a tokenized string of characters.
2. The method of claim 1 , further comprising one or more of: modifying the first substring before tokenizing the accessed string of characters, modifying the second substring before tokenizing the first intermediate string of characters, and modifying the third substring before tokenizing the second intermediate string of characters.
3. The method of claim 2 , wherein one or more of the first substring, the second substring, and the third substring is modified based at least in part on one or more initialization vectors.
4. The method of claim 1 , wherein the second substring comprises fewer characters than the first substring.
5. The method of claim 1 , wherein the second substring comprises at least one character not replaced by the first token.
6. A system, comprising:
a non-transitory computer-readable storage medium storing executable instructions that, when executed, perform steps comprising:
accessing a string of characters;
accessing a first token table, a second token table, and a third token table, each of the first token table, the second token table, and the third token table mapping each of a set of input values to a different token value, wherein the first token table, the second token table, and the third token table comprise different token tables;
tokenizing, by a processor, the accessed string of characters by replacing a first substring of the accessed string of characters with a first token value mapped to a value of the first substring by the first token table to form a first intermediate string of characters;
tokenizing, by the processor, the first intermediate string of characters by replacing a second substring of the first intermediate string of characters with a second token value mapped to a value of the second substring by the second token table to form a second intermediate string of characters; and
tokenizing, by the processor, the second intermediate string of characters by replacing a third substring of the second intermediate string of characters with a third token value mapped to a value of the third substring by the third token table to form a tokenized string of characters; and
a hardware processor configured to execute the instructions.
7. The system of claim 6 , where the instructions, when executed, perform further steps comprising one or more of: modifying the first substring before tokenizing the accessed string of characters, modifying the second substring before tokenizing the first intermediate string of characters, and modifying the third substring before tokenizing the second intermediate string of characters.
8. The system of claim 7 , wherein one or more of the first substring, the second substring, and the third substring is modified based at least in part on one or more initialization vectors.
9. The system of claim 6 , wherein the second substring comprises fewer characters than the first substring.
10. The method of claim 6 , wherein the second substring comprises at least one character not replaced by the first token.
11. A method of tokenization, comprising:
accessing a string of characters;
accessing a first token table and a second token table, each of the first token table and the second token table mapping each of a set of input values to a different token value, the first token table different than the second token table;
tokenizing, by a processor, the accessed string of characters by replacing a first substring of the string of characters with a first token value mapped to a value of the first substring by the first token table to form an intermediate string of characters; and
tokenizing, by the processor, the intermediate string of characters by replacing a second substring of the intermediate string of characters contained entirely within the first substring with a second token value mapped to a value of the second substring by the second token table to form a tokenized string of characters.
12. The method of claim 11 , further comprising one or more of: modifying the first substring before tokenizing the accessed string of characters and modifying the second substring before tokenizing the intermediate string of characters.
13. The method of claim 12 , wherein the first substring or the second substring are modified based at least in part on one or more initialization vectors.
14. The method of claim 13 , wherein modifying a substring comprises adding the substring to one or more initialization vectors.
15. The method of claim 11 , wherein the second substring comprises fewer characters than the first substring.
16. A system, comprising:
a non-transitory computer-readable storage medium storing executable instructions that, when executed, perform steps comprising:
accessing a string of characters;
accessing a first token table and a second token table, each of the first token table and the second token table mapping each of a set of input values to a different token value, the first token table different than the second token table;
tokenizing the accessed string of characters by replacing a first substring of the string of characters with a first token value mapped to a value of the first substring by the first token table to form an intermediate string of characters; and
tokenizing the intermediate string of characters by replacing a second substring of the intermediate string of characters contained entirely within the first substring with a second token value mapped to a value of the second substring by the second token table to form a tokenized string of characters; and
a processor configured to execute the instructions.
17. The system of claim 16 , wherein the instructions, when executed, perform further steps comprising one or more of: modifying the first substring before tokenizing the accessed string of characters and modifying the second substring before tokenizing the intermediate string of characters.
18. The system of claim 17 , wherein the first substring or the second substring are modified based at least in part on one or more initialization vectors.
19. The system of claim 18 , wherein modifying a substring comprises adding the substring to one or more initialization vectors.
20. The system of claim 16 , wherein the second substring comprises fewer characters than the first substring.