Secure multi-party reach and frequency estimation
Systems and methods for generating min-increment counting bloom filters to determine count and frequency of device identifiers and attributes in a networking environment are disclosed. The system can maintain a set of data records including device identifiers and attributes associated with device in a network. The system can generate a vector comprising coordinates corresponding to counter registers. The system can identify hash functions to update a counting bloom filter. The system can hash the data records to extract index values pointing to a set of counter registers. The system can increment the positions in the min-increment counting bloom filter corresponding to the minimum values of the counter registers. The system can obtain an aggregated public key comprising a public key. The system can encrypt the counter registers using the aggregated shared key to generate an encrypted vector. The system can transmit the encrypted vector to a networked worker computing device.
1. A method of secure and computationally efficient estimation of a number of members in a multiset having attributes that satisfy target criteria, comprising:
receiving, by an aggregator computing device comprising one or more processors and a memory, an encrypted data structure from a worker computing device;
constructing, by the aggregator computing device, a histogram data structure using the encrypted data structure;
estimating, by the aggregator computing device and using the histogram data structure, a frequency value representing a number of device identifiers associated with an attribute that satisfies a target criteria; and
transmitting, by the aggregator computing device, the frequency value to a publisher computing device.
2. The method of claim 1 , wherein constructing the histogram data structure further comprises:
transmitting, by the aggregator computing device, the encrypted data structure to a second worker computing device;
receiving, by the aggregator computing device, a partially decrypted data structure from the second worker computing device;
decrypting, by the aggregator computing device, the partially decrypted data structure using a second decryption key to generate a decrypted data structure; and
constructing, by the aggregator computing device, the histogram using the decrypted data structure.
3. The method of claim 1 , further comprising:
receiving, by the aggregator computing device, a plurality of encrypted data structures from a respective plurality of publisher computing devices;
combining, by the aggregator computing device, the plurality of encrypted data structures into a combined encrypted data structure; and
transmitting, by the aggregator computing device, the combined encrypted data structure to the worker computing device.
4. The method of claim 3 , wherein combining the plurality of encrypted data structures further comprises multiplying, by the aggregator computing device, a first encrypted data structure of the plurality of encrypted data structures with a second encrypted data structure of the plurality of encrypted data structures to combine the plurality of encrypted data structures using homomorphic addition.
5. The method of claim 3 , further comprising:
receiving, by the aggregator computing device, a first encrypted noise array from the worker computing device;
receiving, by the aggregator computing device, a second encrypted noise array from a second worker computing device;
aggregating, by the aggregator computing device, the first encrypted noise array and the second encrypted noise array to generate an aggregated noise array; and
appending, by the aggregator computing device, the aggregated noise array to the combined encrypted data structure.
6. The method of claim 1 , further comprising:
determining, by the aggregator computing device, an average noise value; and
subtracting, by the aggregator computing device, the average noise value from at least one value of the histogram data structure to remove noise from the histogram data structure.
7. The method of claim 1 , wherein the encrypted data structure is a set of noised matrices, each of the noised matrices corresponding to a respective one of a set of publisher computing devices, and wherein constructing the histogram data structure further comprises:
estimating, by the aggregator computing device, an estimated cardinality value for each noised matrix of the set of noised matrices;
ranking, by the aggregator computing device, the set of noised matrices based on the estimated cardinality value for each noised matrix of the set of noised matrices;
selecting, by the aggregator computing device, a first subset of the set of noised matrices and a second subset of the set of noised matrices based on the ranking; and
constructing, by the aggregator computing device, the histogram data structure based on the first subset of the set of noised matrices and the second subset of the set of noised matrices.
8. The method of claim 7 , wherein estimating the cardinality value for each noised matrix comprises:
identifying, by the aggregator computing device, a row of the noised matrix that includes a number of zeros that is less than or equal to half the number of positions in the row of the noised matrix, wherein the row of the noised matrix corresponds to a row index value; and
calculating, by the aggregator computing device, the estimated cardinality value of the noised matrix using the row index value.
9. The method of claim 7 , wherein ranking the set of noised matrices further comprises:
determining, by the aggregator computing device, that the estimated cardinality value of a first noised matrix of the set of noised matrices is greater than the estimated cardinality value of a second noised matrix of the set of noised matrices;
assigning, by the aggregator computing device, a first rank value to the first noised matrix that is greater than a second rank value; and
assigning, by the aggregator computing device, the second rank value to the second noised matrix.
10. The method of claim 7 , wherein constructing the histogram data structure further comprises:
generating, by the aggregator computing device, based on a first Monte-Carlo sampling of the first subset of noised matrices, a first union vector;
generating, by the aggregator computing device, based on a second Monte-Carlo sampling of the second subset of noised matrices, a second union vector;
combining, by the aggregator computing device, the first union vector and the second vector into a combined union data structure; and
constructing, by the aggregator computing device, the histogram data structure using the combined union data structure.
11. A system for secure and computationally efficient estimation of a number of members in a multiset having attributes that satisfy target criteria, comprising:
an aggregator computing device comprising one or more processors and a memory, the aggregator computing device configured to:
receive an encrypted data structure from a worker computing device;
construct a histogram data structure using the encrypted data structure;
estimate, using the histogram data structure, a frequency value representing a number of device identifiers associated with an attribute that satisfies a target criteria; and
transmit the frequency value to a publisher computing device.
12. The system of claim 11 , wherein the aggregator computing device is further configured to:
transmit the encrypted data structure to a second worker computing device;
receive a partially decrypted data structure from the second worker computing device;
decrypt the partially decrypted data structure using a second decryption key to generate a decrypted data structure; and
construct the histogram data structure using the decrypted data structure.
13. The system of claim 11 , wherein the aggregator computing device is further configured to:
receive a plurality of encrypted data structures from a respective plurality of publisher computing devices;
combine the plurality of encrypted data structures into a combined encrypted data structure; and
transmit the combined encrypted data structure to the worker computing device.
14. The system of claim 13 , wherein the aggregator computing device is further configured to multiply a first encrypted data structure of the plurality of encrypted data structures with a second encrypted data structure of the plurality of encrypted data structures to combine the plurality of encrypted data structures using homomorphic addition.
15. The system of claim 13 , wherein the aggregator computing device is further configured to:
receive a first encrypted noise array from the worker computing device;
receive a second encrypted noise array from a second worker computing device;
aggregate the first encrypted noise array and the second encrypted noise array to generate an aggregated noise array; and
append the aggregated noise array to the combined encrypted data structure.
16. The system of claim 11 , wherein the aggregator computing device is further configured to:
determine an average noise value; and
subtract the average noise value from at least one value of the histogram data structure to remove noise from the histogram data structure.
17. The system of claim 11 , wherein the encrypted data structure is a set of noised matrices, each of the noised matrices corresponding to a respective one of a set of publisher computing devices, and wherein the aggregator computing device is further configured to:
estimate an estimated cardinality value for each noised matrix of the set of noised matrices;
rank the set of noised matrices based on the estimated cardinality value for each noised matrix of the set of noised matrices;
select a first subset of the set of noised matrices and a second subset of the set of noised matrices based on the ranking; and
construct the histogram data structure based on the first subset of the set of noised matrices and the second subset of the set of noised matrices.
18. The system of claim 17 , wherein the aggregator computing device is further configured to:
identify a row of the noised matrix that includes a number of zeros that is less than or equal to half the number of positions in the row of the noised matrix, wherein the row of the noised matrix corresponds to a row index value; and
calculate the estimated cardinality value of the noised matrix using the row index value.
19. The system of claim 17 , wherein the aggregator computing device is further configured to:
determine that the estimated cardinality value of a first noised matrix of the set of noised matrices is greater than the estimated cardinality value of a second noised matrix of the set of noised matrices;
assign a first rank value to the first noised matrix that is greater than a second rank value; and
assign the second rank value to the second noised matrix.
20. The system of claim 17 , wherein the aggregator computing device is further configured to:
generate based on a first Monte-Carlo sampling of the first subset of noised matrices, a first union vector;
generate, based on a second Monte-Carlo sampling of the second subset of noised matrices, a second union vector;
combine the first union vector and the second vector into a combined union data structure; and
construct the histogram data structure using the combined union data structure.