Parallel tokenization of floating point information in a distributed network environment
Data in various formats can be protected in a distributed tokenization environment. Examples of such formats include date and time data, decimal data, and floating point data. Such data can tokenized by a security device that instantiates a number of tokenization pipelines for parallel tokenization of the data. Characteristics of such data can be used to tokenize the data. For instance, token tables specific to the data format can be used to tokenized the data. Likewise, a type, order, or configuration of the operations within each tokenization pipeline can be selected based on the data format or characteristics of the data format. Each tokenization pipeline performs a set of encoding or tokenization operations in parallel and based at least in part on a value received from another tokenization pipeline. The tokenization pipeline outputs are combined, producing tokenized data, which can be provided to a remote system for storage or processing.
1. A method comprising:
receiving, at a local computing system, a string of characters in a floating point format, the string of characters comprising a significand portion, a base portion, and an exponent portion;
querying, by the local computing system, a token server using the significand portion to access a first set of token tables and using the exponent portion to access a second set of token tables;
instantiating, by the local computing system, a significand tokenization pipeline and an exponent tokenization pipeline, wherein:
the significand tokenization pipeline is configured to perform one or more sequential tokenization operations on the significand portion using the first set of token tables and based on a sign of the significand portion to produce a tokenized significand portion of characters, wherein the significand tokenization pipeline includes at least one tokenization engine and at least one non-tokenization processing engine,
in parallel with the significand tokenization pipeline, the exponent tokenization pipeline is configured to perform one or more sequential tokenization operations on the exponent portion using the second set of token tables and based on a sign of the exponent portion to produce a tokenized exponent portion of characters, wherein the exponent tokenization pipeline includes at least one tokenization engine and at least one non-tokenization processing engine,
one or more operations of the at least one non-tokenization processing engine of the significand tokenization pipeline are based on an output from one or more operations of the at least one tokenization engine of the exponent tokenization pipeline, and one or more operations of the at least one non-tokenization engine of the exponent tokenization pipeline are based on an output from one or more operations of the significand tokenization pipeline; and
producing a protected parallel-tokenized floating point set of characters by combining the tokenized significand portion of characters, the base portion, and the tokenized exponent portion of characters; and
providing, by the local computing system, the protected parallel-tokenized floating point set of characters to a remote computing system.
2. The method of claim 1 , wherein the exponent tokenization pipeline is configured to delay the performance of a first tokenization operation until the significand tokenization pipeline completes the performance of a second tokenization operation.
3. The method of claim 1 , wherein the significand portion comprises a first set of characters, and wherein the exponent portion comprises a second set of characters.
4. The method of claim 3 , where the first set of token tables each map input values of a length equal to a length of the first set of characters to different token values.
5. The method of claim 4 , where the second set of token tables each map input values of a length equal to a length of the second set of characters to different token values.
6. The method of claim 3 , wherein a length of the first set of characters is different from a length of the second set of characters.
7. The method of claim 1 , wherein the token server is located remotely from the local computing system and the remote computing system.
8. A non-transitory computer-readable storage medium storing executable instructions that, when executed by a hardware processor, cause the hardware processor to perform steps comprising:
receiving, at a local computing system, a string of characters in a floating point format, the string of characters comprising a significand portion, a base portion, and an exponent portion;
querying, by the local computing system, a token server using the significand portion to access a first set of token tables and using the exponent portion to access a second set of token tables;
instantiating, by the local computing system, a significand tokenization pipeline and an exponent tokenization pipeline, wherein:
the significand tokenization pipeline is configured to perform one or more sequential tokenization operations on the significand using the first set of token tables and based on a sign of the significand portion to produce a tokenized significand portion of characters, wherein the significand tokenization pipeline includes at least one tokenization engine and at least one non-tokenization processing engine,
in parallel with the significand tokenization pipeline, the exponent tokenization pipeline is configured to perform one or more sequential tokenization operations on the exponent portion using the second set of token tables and based on a sign of the exponent portion to produce a tokenized exponent portion of characters, wherein the exponent tokenization pipeline includes at least one tokenization engine and at least one non-tokenization processing engine,
one or more operations of the at least one non-tokenization processing engine of the significand tokenization pipeline are based on an output from one or more operations of the at least one tokenization engine of the exponent tokenization pipeline, and one or more operations of the at least one non-tokenization engine of the exponent tokenization pipeline are based on an output from one or more operations of the significand tokenization pipeline; and
producing a protected parallel-tokenized floating point set of characters by combining the tokenized significand portion of characters, the base portion, and the tokenized exponent portion of characters; and
providing, by the local computing system, the protected parallel-tokenized floating point set of characters to a remote computing system.
9. The non-transitory computer-readable storage medium of claim 8 , wherein the exponent tokenization pipeline is configured to delay the performance of a first tokenization operation until the significand tokenization pipeline completes the performance of a second tokenization operation.
10. The non-transitory computer-readable storage medium of claim 8 , wherein the significand portion comprises a first set of characters, and wherein the exponent portion comprises a second set of characters.
11. The non-transitory computer-readable storage medium of claim 10 , where the first set of token tables each map input values of a length equal to a length of the first set of characters to different token values.
12. The non-transitory computer-readable storage medium of claim 11 , where the second set of token tables each map input values of a length equal to a length of the second set of characters to different token values.
13. The non-transitory computer-readable storage medium of claim 10 , wherein a length of the first set of characters is different from a length of the second set of characters.
14. The non-transitory computer-readable storage medium of claim 8 , wherein the token server is located remotely from the local computing system and the remote computing system.
15. A system comprising:
a hardware processor; and
a non-transitory computer-readable storage medium storing executable instructions that, when executed by the hardware processor, cause the hardware processor to perform steps comprising:
receiving, at a local computing system, a string of characters in a floating point format, the string of characters comprising a significand portion, a base portion, and an exponent portion;
querying, by the local computing system, a token server using the significand portion to access a first set of token tables and using the exponent portion to access a second set of token tables;
instantiating, by the local computing system, a significand tokenization pipeline and an exponent tokenization pipeline, wherein:
the significand tokenization pipeline is configured to perform one or more sequential tokenization operations on the significand using the first set of token tables and based on a sign of the significand portion to produce a tokenized significand portion of characters, wherein the significand tokenization pipeline includes at least one tokenization engine and at least one non-tokenization processing engine,
in parallel with the significand tokenization pipeline, the exponent tokenization pipeline is configured to perform one or more sequential tokenization operations on the exponent portion using the second set of token tables and based on a sign of the exponent portion to produce a tokenized exponent portion of characters, wherein the exponent tokenization pipeline includes at least one tokenization engine and at least one non-tokenization processing engine,
one or more operations of the at least one non-tokenization processing engine of the significand tokenization pipeline are based on an output from one or more operations of the at least one tokenization engine of the exponent tokenization pipeline, and one or more operations of the at least one non-tokenization engine of the exponent tokenization pipeline are based on an output from one or more operations of the significand tokenization pipeline; and
producing a protected parallel-tokenized floating point set of characters by combining the tokenized significand portion of characters, the base portion, and the tokenized exponent portion of characters; and
providing, by the local computing system, the protected parallel-tokenized floating point set of characters to a remote computing system.
16. The system of claim 15 , wherein the exponent tokenization pipeline is configured to delay the performance of a first tokenization operation until the significand tokenization pipeline completes the performance of a second tokenization operation.
17. The system of claim 15 , wherein the significand portion comprises a first set of characters, and wherein the exponent portion comprises a second set of characters.
18. The system of claim 17 , where the first set of token tables each map input values of a length equal to a length of the first set of characters to different token values.
19. The system of claim 18 , where the second set of token tables each map input values of a length equal to a length of the second set of characters to different token values.
20. The system of claim 17 , wherein a length of the first set of characters is different from a length of the second set of characters.