Scalable and differentially private distributed aggregation
An encoding process performed by a computing device (e.g., a user's private device) can include obtaining private data that includes a private value. According to an aspect of the present disclosure, the computing device can produce a plurality of messages that respectively comprise a plurality of message values, where a total sum of the plurality of message values approximates the private value, and where at least one of the plurality of message values is randomly selected. The device can provide the plurality of messages for aggregation with a plurality of additional messages respectively generated for a plurality of additional private values. For example, the messages can be transmitted to a shuffler model configured to shuffle the plurality of messages with the plurality of additional messages.
1. A computer-implemented method to enable privacy-preserving aggregation of private data, the method comprising:
obtaining, by one or more computing devices, private data comprising a private value;
producing, by the one or more computing devices based at least in part on the private value, a plurality of messages that respectively comprise a plurality of message values, wherein the plurality of message values together encode the private value for differentially private aggregation, wherein producing the plurality of messages comprises:
randomly selecting, by the one or more computing devices, a message value for each of one or more first messages of the plurality of messages;
determining, by the one or more computing devices, an intermediate sum of the message values of the one or more first messages; and
setting, by the one or more computing devices, a message value of a final message of the plurality of messages equal to the private value minus the intermediate sum modulo a first parameter value; and
transmitting, by the one or more computing devices over a communication channel, the plurality of messages, including the final message and the one or more first messages, for differentially private aggregation of the private value with a plurality of additional private values based on a plurality of additional messages respectively generated for the plurality of additional private values.
2. The computer-implemented method of claim 1 , wherein randomly selecting, by the one or more computing devices, a message value comprises:
uniformly and randomly sampling, by the one or more computing devices, one of a plurality of available values to serve as the message value for such first message.
3. The computer-implemented method of claim 2 , wherein:
the plurality of available values comprises a set of integers extending from zero to a sample control parameter value minus one; and
the first parameter value is equal to the sample control parameter value.
4. The computer-implemented method of claim 2 , wherein a number of the one or more first iterations is controlled by a message control parameter value.
5. The computer-implemented method of claim 1 , wherein:
the private value comprises a scaled private value produced by scaling an unscaled private value; and
obtaining, by one or more computing devices, the private data comprising the private value comprises scaling, by the one or more computing devices, the unscaled private value by a scaling control parameter value to obtain the scaled private value.
6. The computer-implemented method of claim 1 , wherein:
the private value comprises a normalized private value produced by normalizing a raw private value; and
obtaining, by one or more computing devices, the private data comprising the private value comprises normalizing, by the one or more computing devices, the raw private value according to an expected maximum private value.
7. The computer-implemented method of claim 6 , further comprising scaling, by the one or more computing devices, the normalized private value by a scaling control parameter value to obtain a scaled private value.
8. The computer-implemented method of claim 1 , wherein:
the private value comprises a noised private value produced by adding noise to a raw private value; and
obtaining, by one or more computing devices, the private data comprising the private value comprises pre-randomizing, by the one or more computing devices, the raw private value according to a shared noise probability to obtain the noised private value.
9. The computer-implemented method of claim 1 , wherein one or more of the sampling control parameter value, the scaling control parameter value, and the message control parameter value comprises a user-specified hyperparameter or a learned hyperparameter.
10. The computer-implemented method of claim 1 , wherein one or more of the sampling control parameter value, the scaling control parameter value, and the message control parameter value is greater than or equal to four.
11. The computer-implemented method of claim 1 wherein providing, by the one or more computing devices, the plurality of messages for aggregation comprises transmitting, by the one or more computing devices, the plurality of messages to a shuffler model configured to shuffle the plurality of messages with the plurality of additional messages.
12. The computer-implemented method of claim 1 , wherein the one or more computing devices consist of a user device.
13. The computer-implemented method of claim 1 , wherein the private value comprises one or more of:
an update value for a parameter of a machine-learned model;
a heavy hitter value;
an entropy value;
a quantization value; or
a support size value.
14. The computer-implemented method of claim 1 , further comprising:
encrypting, by the one or more computing devices, at least one of the plurality of messages with a public key associated with a shuffler model configured to shuffle the plurality of messages.