Automatic detection and protection of personally identifiable information

1. A method for detecting personally identifiable information, comprising:

retrieving a user-specific dataset from a multi-user dataset;

filtering the user-specific dataset to create a user-specific data subset;

determining a user frequency for each user-specific token of a plurality of user-specific tokens in the user-specific data subset;

determining a multi-user frequency for each user-specific token of the plurality of user-specific tokens in the multi-user dataset, wherein each multi-user frequency is determined based on a frequency of a respective user-specific token across the multi-user dataset;

computing a frequency ratio, based on the user frequency for each user-specific token and the multi-user frequency for each user-specific token, for each user-specific token of the plurality of user-specific tokens; and

protecting each user-specific token whose frequency ratio is above a frequency ratio threshold.

2. The method of claim 1 , wherein protecting each user-specific token whose frequency ratio is above the frequency ratio threshold comprises encrypting each user-specific token whose frequency ratio is above the frequency ratio threshold in a production dataset different from the multi-user dataset.

3. The method of claim 1 , further comprising: creating the plurality of user-specific tokens based on the user-specific data subset.

4. The method of claim 3 , wherein filtering the user-specific dataset comprises: removing each user-specific token in the user-specific dataset whose user frequency is lower than a threshold frequency.

5. The method of claim 3 , wherein filtering the user-specific dataset comprises: removing each user-specific token in the user-specific dataset that matches a token in an insignificant token dictionary.

6. The method of claim 3 , wherein filtering the user-specific dataset comprises: removing encrypted text from the user-specific dataset.

7. The method of claim 1 , further comprising:

scaling the user frequency for each user-specific token of the plurality of user-specific tokens in the user-specific data subset based on a total number of tokens in the user-specific data subset; and

scaling the multi-user frequency for each user-specific token of the plurality of user-specific tokens in the multi-user dataset based on a total number of tokens in the multi-user dataset.

8. A personally identifiable information detection system comprising:

a memory comprising computer-executable instructions;

a processor configured to execute the computer-executable instructions and cause the personally identifiable information detection system to perform a method of detecting personally identifiable information, the method comprising:

retrieving a user-specific dataset from a multi-user dataset;

filtering the user-specific dataset to create a user-specific data subset;

determining a user frequency for each user-specific token of a plurality of user-specific tokens in the user-specific data subset;

determining a multi-user frequency for each user-specific token of the plurality of user-specific tokens in the multi-user dataset, wherein each multi-user frequency is determined based on a frequency of a respective user-specific token across the multi-user dataset;

computing a frequency ratio, based on the user frequency for each user-specific token and the multi-user frequency for each user-specific token, for each user-specific token of the plurality of user-specific tokens; and

protecting each user-specific token whose frequency ratio is above a frequency ratio threshold.

9. The personally identifiable information detection system of claim 8 , wherein protecting each user-specific token whose frequency ratio is above the frequency ratio threshold comprises encrypting each user-specific token whose frequency ratio is above the frequency ratio threshold in a production dataset different from the multi-user dataset.

10. The personally identifiable information detection system of claim 8 , wherein the method further comprises: creating the plurality of user-specific tokens based on the user-specific data subset.

11. The personally identifiable information detection system of claim 10 , wherein filtering the user-specific dataset comprises: removing each user-specific token in the user-specific dataset whose user frequency is lower than a threshold frequency.

12. The personally identifiable information detection system of claim 10 , wherein filtering the user-specific dataset comprises: removing each user-specific token in the user-specific dataset that matches a token in an insignificant token dictionary.

13. The personally identifiable information detection system of claim 10 , wherein filtering the user-specific dataset comprises: removing encrypted text from the user-specific dataset.

14. The personally identifiable information detection system of claim 8 , wherein the method further comprises:

scaling the user frequency for each user-specific token of the plurality of user-specific tokens in the user-specific data subset based on a total number of tokens in the user-specific data subset; and

scaling the multi-user frequency for each user-specific token of the plurality of user-specific tokens in the multi-user dataset based on a total number of tokens in the multi-user dataset.

15. A non-transitory computer-readable medium comprising instructions that, when executed by a processor of a processing system, cause the processing system to perform a method for detecting personally identifiable information, the method comprising:

retrieving a user-specific dataset from a multi-user dataset;

filtering the user-specific dataset to create a user-specific data subset;

determining a user frequency for each user-specific token of a plurality of user-specific tokens in the user-specific data subset;

determining a multi-user frequency for each user-specific token of the plurality of user-specific tokens in the multi-user dataset, wherein each multi-user frequency is determined based on a frequency of a respective user-specific token across the multi-user dataset;

computing a frequency ratio, based on the user frequency for each user-specific token and the multi-user frequency for each user-specific token, for each user-specific token of the plurality of user-specific tokens; and

protecting each user-specific token whose frequency ratio is above a frequency ratio threshold.

16. The non-transitory computer-readable medium of claim 15 , wherein protecting each user-specific token whose frequency ratio is above the frequency ratio threshold comprises encrypting each user-specific token whose frequency ratio is above the frequency ratio threshold in a production dataset different from the multi-user dataset.

17. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises: creating the plurality of user-specific tokens based on the user-specific data subset.

18. The non-transitory computer-readable medium of claim 17 , wherein filtering the user-specific dataset comprises at least one of:

removing each user-specific token in the user-specific dataset whose user frequency is lower than a threshold frequency; or

removing each user-specific token in the user-specific dataset that matches a token in an insignificant token dictionary.

19. The non-transitory computer-readable medium of claim 17 , wherein filtering the user-specific dataset comprises: removing encrypted text from the user-specific dataset.

20. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises:

scaling the user frequency for each user-specific token of the plurality of user-specific tokens in the user-specific data subset based on a total number of tokens in the user-specific data subset; and

scaling the multi-user frequency for each user-specific token of the plurality of user-specific tokens in the multi-user dataset based on a total number of tokens in the multi-user dataset.