Practical private algorithms for robust statistics
Embodiments described herein provide a privacy mechanism to protect user data when transmitting the data to a server that estimates a p-th frequency moment, F p for p∈[1, 2] and p low-rank approximation for p∈[1, 2). The privacy mechanism uses an encode-shuffle then analyze (ESA) framework that provides a compromise between the central and local model of privacy.
1. A client computing device comprising:
one or more memory devices, the one or more memory devices to store executable instructions and a dataset having a universe of values, the universe of values having a first number of values;
one or more processors configured to execute the instructions stored on the memory device, wherein the instructions cause the one or more processors to implement a local randomizer for the dataset, wherein the local randomizer has a specified privacy parameter and the instructions cause the one or more processors to:
generate a matrix based on independent and identically distributed samples of a p-stable distribution of the dataset, wherein the matrix includes the first number of columns, a second number of rows, and the first number is larger than the second number;
generate a sketch having a size based on the second number;
apply a first randomization function to coordinates of the sketch to generate a randomized sketch; and
transmit a report to a server, the report including the randomized sketch, wherein the randomized sketch enables a privatized estimation of a frequency moment or a low-rank approximation based on the dataset; and wherein the randomized sketch enables estimation of the frequency moment via a non-private estimator while maintaining differential privacy for the dataset, and the privatized estimation of the frequency moment is (ε, δ)-differentially private via a shuffle model of privacy.
2. The client computing device as in claim 1 , wherein the frequency moment is a first frequency moment and indicates an estimate of a number of users that contributed to the dataset.
3. The client computing device as in claim 1 , the instructions cause the one or more processors to:
generate multiple randomized sketches based on the matrix and the first randomization function, wherein the report transmitted to the server includes the multiple randomized sketches and the report enables the privatized estimation of the low-rank approximation for the dataset.
4. The client computing device as in claim 3 , wherein the multiple randomized sketches include a randomized sketch of a column space, a randomized sketch of a row space, and a randomized sketch of values of the matrix.
5. The client computing device as in claim 4 , wherein the privatized estimation of the low-rank approximation for the dataset is (ε, δ)-differentially private via the shuffle model of privacy.
6. A server computing device comprising:
one or more memory devices, the one or more memory devices configured to store executable instructions;
one or more processors configured to execute the instructions stored on the memory device, wherein the instructions cause the one or more processors to implement an analyzer to estimate a frequency moment or a low-rank approximation of a dataset and the instructions cause the one or more processors to:
receive a report from a client device, the report including a randomized sketch, the randomized sketch generated by the client device based on the dataset having a universe of values, the universe of values having a first number of values, and the randomized sketch having a size based on a second number that is less than the first number, wherein the randomized sketch was generated from a randomization matrix having the first number of columns and the second number of rows;
combine the randomized sketch from the client device with a set of randomized sketches received from a plurality of other client devices to generate a cumulative sketch; and
estimate the frequency moment or the low-rank approximation of the dataset based on the cumulative sketch, wherein the frequency moment is estimated via a first non-private estimator while maintaining differential privacy for the dataset, and wherein the estimate of the frequency moment is (ε, δ)-differentially private via a shuffle model of privacy.
7. The server computing device as in claim 6 , wherein the frequency moment is a first frequency moment and indicates an estimate of a number of users that contributed to the dataset.
8. The server computing device as in claim 6 , wherein the report includes multiple randomized sketches, and wherein the report enables the server computing device to estimate a privatized low-rank approximation for the dataset.
9. The server computing device as in claim 8 , wherein the multiple randomized sketches include a randomized sketch of a column space, a randomized sketch of a row space, and a randomized sketch of values of the randomization matrix.
10. The server computing device as in claim 9 , wherein the estimate of the low-rank approximation for the dataset is (ε, δ)-differentially private via the shuffle model of privacy.
11. A computer readable medium storing instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform operations comprising:
generating a matrix based on independent and identically distributed samples of a p-stable distribution of a dataset stored on the electronic device, the dataset having a universe of values, the universe of values having a first number of values, wherein the matrix includes the first number of columns, a second number of rows, and the first number is larger than the second number;
generating a sketch having a size based on the second number;
applying a first randomization function to coordinates of the sketch to generate a randomized sketch; and
transmitting a report to a server, the report including the randomized sketch, wherein the randomized sketch enables a privatized estimation of a frequency moment or a low-rank approximation based on the dataset, wherein the randomized sketch enables estimation of the frequency moment via a non-private estimator while maintaining differential privacy for the dataset, and wherein the privatized estimation of the frequency moment is (ε, δ)-differentially private via a shuffle model of privacy.
12. The computer readable medium of claim 11 , wherein the frequency moment is a first frequency moment and indicates an estimate of a number of users that contributed to the dataset.
13. The computer readable medium of claim 11 , wherein the operations further comprise:
generating multiple randomized sketches based on the matrix and the first randomization function, wherein the report transmitted to the server includes the multiple randomized sketches and the report enables the privatized estimation of the low-rank approximation for the dataset.
14. The computer readable medium of claim 13 , wherein the multiple randomized sketches include a randomized sketch of a column space, a randomized sketch of a row space, and a randomized sketch of values of the matrix.
15. The computer readable medium of claim 13 , wherein the privatized estimation of the low-rank approximation for the dataset is (ε, δ)-differentially private via the shuffle model of privacy.
16. A method comprising performing, by an electronic device:
generating a matrix based on independent and identically distributed samples of a p-stable distribution of a dataset stored on the electronic device, the dataset having a universe of values, the universe of values having a first number of values, wherein the matrix includes the first number of columns, a second number of rows, and the first number is larger than the second number;
generating a sketch having a size based on the second number;
applying a first randomization function to coordinates of the sketch to generate a randomized sketch; and
transmitting a report to a server, the report including the randomized sketch, wherein the randomized sketch enables a privatized estimation of a frequency moment or a low-rank approximation based on the dataset, wherein the randomized sketch enables estimation of the frequency moment via a non-private estimator while maintaining differential privacy for the dataset, and wherein the privatized estimation of the frequency moment is (ε, δ)-differentially private via a shuffle model of privacy.
17. The method of claim 16 , wherein the frequency moment is a first frequency moment and indicates an estimate of a number of users that contributed to the dataset.
18. The method of claim 16 , further comprising:
generating multiple randomized sketches based on the matrix and the first randomization function, wherein the report transmitted to the server includes the multiple randomized sketches and the report enables the privatized estimation of the low-rank approximation for the dataset.
19. The method of claim 18 , wherein the multiple randomized sketches include a randomized sketch of a column space, a randomized sketch of a row space, and a randomized sketch of values of the matrix.
20. The method of claim 18 , wherein the privatized estimation of the low-rank approximation for the dataset is (ε, δ)-differentially private via the shuffle model of privacy.
21. A method comprising performing, by a server computing device:
receive a report from a client device, the report including a randomized sketch, the randomized sketch generated by the client device based on a dataset having a universe of values, the universe of values having a first number of values, and the randomized sketch having a size based on a second number that is less than the first number, wherein the randomized sketch was generated from a randomization matrix having the first number of columns and the second number of rows;
combine the randomized sketch from the client device with a set of randomized sketches received from a plurality of other client devices to generate a cumulative sketch; and
estimate a frequency moment or a low-rank approximation of the dataset based on the cumulative sketch, wherein the frequency moment is estimated via a first non-private estimator while maintaining differential privacy for the dataset, and wherein the estimate of the frequency moment is (ε, δ)-differentially private via a shuffle model of privacy.
22. The method of claim 21 , wherein the frequency moment is a first frequency moment and indicates an estimate of a number of users that contributed to the dataset.
23. The method of claim 21 , wherein the report includes multiple randomized sketches, and wherein the report enables the server computing device to estimate a privatized low-rank approximation for the dataset.
24. The method of claim 23 , wherein the multiple randomized sketches include a randomized sketch of a column space, a randomized sketch of a row space, and a randomized sketch of values of the randomization matrix.
25. The method of claim 23 , wherein the estimate of the low-rank approximation for the dataset is (ε, δ)-differentially private via the shuffle model of privacy.