Big data distributed processing and secure data transferring with obfuscation

1. A computer-implemented method of in-flight data masking and on-demand encryption of big data on a network comprising the steps of:

authenticating, by an obfuscation computer machine, a request for a big data dataset based on credentials received from a source,

said obfuscation computer machine coupled to the network,

said obfuscation computer machine containing computer-readable memory,

said request stored in a first sector of the computer-readable memory;

said big data dataset stored in a data store coupled to the network, said big data dataset including sensitive information and non-sensitive information;

auto provisioning, by the obfuscation computer machine, a profile corresponding to the request, said profile stored in a second sector of the computer-readable memory;

assigning, by the obfuscation computer machine, access rights for the request based on the profile, said access rights stored in a third sector of the computer-readable memory;

defining, by the obfuscation computer machine, server configuration and data connection properties for the data store containing the big data dataset, said server configuration and said data connection properties stored in a fourth sector of computer-readable memory;

establishing, by the obfuscation computer machine, a secure connection from the obfuscation computer machine to the data store based on the server connection and the data connection properties;

registering, by the obfuscation computer machine, at least one multiple data obfuscation type stored in a fifth sector of computer-readable memory;

loading, by the obfuscation computer machine into a sixth sector of computer-readable memory, the big data dataset from the data store on the network;

searching, by the obfuscation computer machine, the big data dataset for the sensitive information to be redacted;

redacting into a sanitized dataset, by the obfuscation computer machine, the sensitive information in the big data dataset based on said at least one multiple data obfuscation type and the non-sensitive information;

storing, by the obfuscation computer machine into a seventh sector of computer-readable memory, the sanitized dataset;

encrypting, by the obfuscation computer machine, the sanitized dataset into encrypted data, said encrypted data stored in an eighth sector of computer-readable memory; and

transmitting, by the obfuscation computer machine, the encrypted data, which can then be decrypted.

2. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 wherein encrypting is performed using data lake encryption.

3. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 wherein the obfuscating computer machine is in an upper lane of a multilevel computing platform.

4. The computer-implemented method of in-flight data masking and on-demand encryption of claim 3 wherein the source is in an upper lane of the multilevel computer platform.

5. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 wherein encrypting is performed using a Base64 algorithm.

6. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 wherein encrypting is performed using a text-to-binary scheme.

7. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 wherein redacting the sensitive information in the big data dataset comprises redacting the sensitive information in the big data dataset using a pre-defined mask.

8. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 wherein redacting the sensitive information in the big data dataset comprises redacting the sensitive information in the big data dataset using random generation.

9. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 wherein redacting the sensitive information in the big data dataset comprises redacting the sensitive information in the big data dataset using regular expression obfuscation.

10. The computer-implemented method of in-flight data masking and on-demand encryption of claim 9 wherein the regular expression obfuscation is used to capture pattern values and replace the pattern values with a replacing value format.

11. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 further comprising the step of cataloging categories of data for obfuscation.

12. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 further comprising the steps of diagnosing, by the obfuscation computer machine, resource allocation and initiating, by the obfuscation computer machine, a RAM rebate.

13. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 further comprising the step of implementing, by the obfuscation computer machine, a fault handling algorithm.

14. The computer-implemented method of in-flight data masking and on-demand encryption of claim 13 further comprising the step of implementing, by the obfuscation computer machine, a fall back controller to stop in-memory batch processing if a fault occurs.

15. The computer-implemented method of in-flight data masking and on-demand encryption of claim 1 further comprising the step of generating, by the obfuscation computer machine, a post process performance report stored in a ninth sector of computer-readable memory.

16. The computer-implemented method of in-flight data masking and on-demand encryption of claim 15 wherein the post process performance report is an obfuscation summary report.

17. The computer-implemented method of in-flight data masking and on-demand encryption of claim 15 wherein the post process performance report is a data forensics report.

18. The computer-implemented method of in-flight data masking and on-demand encryption of claim 15 wherein the post process performance report is a data processing summary report.

19. A non-transitory computer-readable medium with computer-executable instructions stored thereon executed by a processor on an obfuscation computer machine to perform in-flight data masking and on-demand encryption of a big data dataset stored in a data store on a network, said big data dataset including sensitive information and non-sensitive information, said obfuscation computer machine communicatively coupled to the network, comprising:

authentication instructions to authenticate a request for the big data dataset based on credentials received from a source,

auto provision instructions to identify a profile corresponding to the request, said profile stored in a first sector of the computer-readable medium;

access instructions to assign access rights for the request based on the profile, said access rights stored in a second sector of the computer-readable medium;

server configuration and data connection instructions to define connection properties for the data store containing the big data dataset, said server configuration and said data connection properties stored in a third sector of the computer-readable medium,

secure connection instructions to establish a secure connection from the obfuscation computer machine to the data store based on the server connection and the data connection properties;

registration instructions to register at least one multiple data obfuscation type stored in a fourth sector of the computer-readable medium;

load instructions to load the big data dataset from the data store on the network into a fifth sector of the computer-readable medium;

search instructions to search the big data dataset for the sensitive information to be redacted;

redaction instructions to redact into a sanitized dataset the sensitive information in the big data dataset based on said at least one multiple data obfuscation type and the non-sensitive information;

storage instructions to store the sanitized dataset into a sixth sector of the computer-readable medium;

encryption instructions to encrypt the sanitized dataset into encrypted data, said encrypted data stored in a seventh sector of the computer-readable medium; and

transmission instructions to transmit the encrypted data in response to the request, wherein the encrypted data can thereafter be decrypted.

20. A multilevel computing platform for performing in-flight data masking and on-demand encryption of big data on a network comprising:

a data store coupled to the network, said data store containing a big data dataset including sensitive information and non-sensitive information;

an upper lane platform having:

at least one upper processor,

at least one upper communication interface communicatively coupled to the at least one upper processor and the network, and

an upper computer-readable memory communicatively coupled to the at least one upper communication interface, the upper computer-readable memory storing upper computer-executable instructions that, when executed by said at least one upper processor, cause the upper lane platform to:

authenticate a request for the big data dataset received via the upper communication interface from a lower lane platform;

auto provision a profile corresponding to the request;

assign access rights for the request based on the profile;

define server configuration and data connection instructions for the data store containing the big data dataset;

securely connect the upper platform to the data store via the upper communication interface based on the server configuration and the data connection instructions;

register at least one multiple data obfuscation type;

load the big data dataset from the data store into a first upper sector in the upper computer-readable memory via the upper communication interface;

search the big data dataset for the sensitive information to be redacted;

redact into a sanitized dataset the sensitive information in the big data dataset based on said at least one multiple data obfuscation type and the non-sensitive information;

store the sanitized dataset in a second upper sector of the upper computer-readable memory;

encrypt the sanitized dataset into encrypted data in a third upper sector in the upper computer-readable memory;

transmit the encrypted data to the lower lane platform via the upper communication interface; and

the lower lane platform having:

at least one lower processor,

at least one lower communication interface communicatively coupled to the at least one lower processor and the network, and

a lower computer-readable memory communicatively coupled to the at least one lower communication interface, the lower computer-readable memory storing lower computer-executable instructions that, when executed by said at least one lower processor, cause the lower lane platform to:

transmit the request for the big data dataset via the lower communication interface to the upper platform;

receive the encrypted data via the lower communication interface from the upper platform;

store the encrypted data in a first lower sector of the lower computer-readable memory;

decrypt the encrypted data into the sanitized dataset; and

store the sanitized dataset in a second lower sector of the lower computer-readable medium.