Parallel tokenization of decimal information in a distributed network environment
Data in various formats can be protected in a distributed tokenization environment. Examples of such formats include date and time data, decimal data, and floating point data. Such data can tokenized by a security device that instantiates a number of tokenization pipelines for parallel tokenization of the data. Characteristics of such data can be used to tokenize the data. For instance, token tables specific to the data format can be used to tokenized the data. Likewise, a type, order, or configuration of the operations within each tokenization pipeline can be selected based on the data format or characteristics of the data format. Each tokenization pipeline performs a set of encoding or tokenization operations in parallel and based at least in part on a value received from another tokenization pipeline. The tokenization pipeline outputs are combined, producing tokenized data, which can be provided to a remote system for storage or processing.
1. A method comprising:
receiving, at a local computing system, a string of characters in a decimal format, the string of characters comprising a whole number portion and a decimal portion;
querying, by the local computing system, a token server using the whole number portion to access a first set of token tables and using the decimal portion to access a second set of token tables;
instantiating, by the local computing system, a whole number tokenization pipeline and a decimal tokenization pipeline, the whole number tokenization pipeline configured to perform one or more sequential tokenization operations on the whole number portion using the first set of token tables to produce a tokenized whole number set of characters in parallel with the decimal tokenization pipeline configured to perform one or more sequential tokenization operations on the decimal portion using the second set of token tables to produce a tokenized decimal set of characters, wherein a first tokenization operation for the decimal tokenization pipeline is based on an output of a second tokenization operation from the whole number tokenization pipeline; and
combining the tokenized whole number set of characters and the tokenized decimal set of characters to produce a tokenized decimal output and providing, by the local computing system, the tokenized decimal output to a remote computing system.
2. The method of claim 1 , wherein the decimal tokenization pipeline is configured to delay the performance of the first tokenization operation until the whole number tokenization pipeline completes the performance of the second tokenization operation.
3. The method of claim 1 , wherein the whole number portion comprises a first set of characters, and wherein the decimal portion comprises a second set of characters.
4. The method of claim 3 , where the first set of token tables each map input values of a length equal to a length of the first set of characters to different token values.
5. The method of claim 4 , where the second set of token tables each map input values of a length equal to a length of the second set of characters to different token values.
6. The method of claim 3 , wherein a length of the first set of characters is different from a length of the second set of characters.
7. The method of claim 1 , wherein the token server is located remotely from the local computing system and the remote computing system.
8. A non-transitory computer-readable storage medium storing executable instructions that, when executed by a hardware processor, cause the hardware processor to perform steps comprising:
receiving, at a local computing system, a string of characters in a decimal format, the string of characters comprising a whole number portion and a decimal portion;
querying, by the local computing system, a token server using the whole number portion to access a first set of token tables and using the decimal portion to access a second set of token tables;
instantiating, by the local computing system, a whole number tokenization pipeline and a decimal tokenization pipeline, the whole number tokenization pipeline configured to perform one or more sequential tokenization operations on the whole number portion using the first set of token tables to produce a tokenized whole number set of characters in parallel with the decimal tokenization pipeline configured to perform one or more sequential tokenization operations on the decimal portion using the second set of token tables to produce a tokenized decimal set of characters, wherein a first tokenization operation for the decimal tokenization pipeline is based on an output of a second tokenization operation from the whole number tokenization pipeline; and
combining the tokenized whole number set of characters and the tokenized decimal set of characters to produce a tokenized decimal output and providing, by the local computing system, the tokenized decimal output to a remote computing system.
9. The non-transitory computer-readable storage medium of claim 8 , wherein the decimal tokenization pipeline is configured to delay the performance of the first tokenization operation until the whole number tokenization pipeline completes the performance of the second tokenization operation.
10. The non-transitory computer-readable storage medium of claim 8 , wherein the whole number portion comprises a first set of characters, and wherein the decimal portion comprises a second set of characters.
11. The non-transitory computer-readable storage medium of claim 10 , where the first set of token tables each map input values of a length equal to a length of the first set of characters to different token values.
12. The non-transitory computer-readable storage medium of claim 11 , where the second set of token tables each map input values of a length equal to a length of the second set of characters to different token values.
13. The non-transitory computer-readable storage medium of claim 10 , wherein a length of the first set of characters is different from a length of the second set of characters.
14. The non-transitory computer-readable storage medium of claim 8 , wherein the token server is located remotely from the local computing system and the remote computing system.
15. A system comprising:
a hardware processor; and
a non-transitory computer-readable storage medium storing executable instructions that, when executed by the hardware processor, cause the hardware processor to perform steps comprising:
receiving, at a local computing system, a string of characters in a decimal format, the string of characters comprising a whole number portion and a decimal portion;
querying, by the local computing system, a token server using the whole number portion to access a first set of token tables and using the decimal portion to access a second set of token tables;
instantiating, by the local computing system, a whole number tokenization pipeline and a decimal tokenization pipeline, the whole number tokenization pipeline configured to perform one or more sequential tokenization operations on the whole number portion using the first set of token tables to produce a tokenized whole number set of characters in parallel with the decimal tokenization pipeline configured to perform one or more sequential tokenization operations on the decimal portion using the second set of token tables to produce a tokenized decimal set of characters, wherein a first tokenization operation for the decimal tokenization pipeline is based on an output of a second tokenization operation from the whole number tokenization pipeline; and
combining the tokenized whole number set of characters and the tokenized decimal set of characters to produce a tokenized decimal output and providing, by the local computing system, the tokenized decimal output to a remote computing system.
16. The system of claim 15 , wherein the decimal tokenization pipeline is configured to delay the performance of the first tokenization operation until the whole number tokenization pipeline completes the performance of the second tokenization operation.
17. The system of claim 15 , wherein the whole number portion comprises a first set of characters, and wherein the decimal portion comprises a second set of characters.
18. The system of claim 17 , where the first set of token tables each map input values of a length equal to a length of the first set of characters to different token values.
19. The system of claim 18 , where the second set of token tables each map input values of a length equal to a length of the second set of characters to different token values.
20. The system of claim 17 , wherein a length of the first set of characters is different from a length of the second set of characters.