Subject level privacy attack analysis for federated learning
Subject level privacy attack analysis for federated learning may be performed. A request that selects an analysis of one or more inference attacks may be received to determine a presence of data of a subject in a training set of a federated machine learning model. The selected inference attacks may be performed to determine the presence of the data of subject in the training set of the federated machine learning model. Respective success measurements may be generated for the selected inference attacks based on the performance of the selected inference attacks, which may then be provided.
1. A system, comprising:
at least one processor;
a memory, comprising program instructions that when executed by the at least one processor cause the at least one processor to implement a federated machine learning model analysis system, the federated machine learning model analysis system configured to:
receive, via an interface of the federated machine learning model analysis system, a request that selects an analysis of one or more inference attacks to determine a presence of data of a subject in a training set of a federated machine learning model;
access the federated machine learning model to perform the selected one or more inference attacks to determine the presence of the data of the subject in the training set of the federated machine learning model;
analyze respective inferences produced by the federated machine learning model as part of performing the selected one or more inference attacks to determine respective success measurements for the selected one or more inference attacks; and
provide, via the interface, the respective success measurements for the selected one or more inference attacks.
2. The system of claim 1 , wherein one of the selected one or more inference attacks is a subject level membership inference attack that compares respective loss values determined from an application of the federated machine learning model to a subject distribution for the subject with a loss threshold to indicate the presence or absence of the subject distribution.
3. The system of claim 1 , wherein one of the selected one or more inference attacks is a subject level loss across rounds inference attack that determines respective loss values from an application of the federated machine learning model to a subject pool for the subject determined at different respective training rounds to indicate the presence or absence of the subject according to a trend analysis of the respective loss values.
4. The system of claim 1 , wherein one of the selected one or more inference attacks is a subject level loss across neighborhood inference attack that determines respective loss values from an application of the federated machine learning model to samples within a ball region of the subject with respectively added noise to indicate the presence or absence of the subject according to a comparison with a loss threshold.
5. The system of claim 1 , wherein the request that selects the one or more inference attacks further identifies the federated machine learning model.
6. The system of claim 1 , wherein the respective success measurements for the selected one or more inference attacks are provided as part of a result that also comprises one or more remedial actions.
7. The system of claim 6 , wherein the one or more remedial actions include a change to a configuration of the federated machine learning model.
8. A method, comprising:
receiving, via an interface of a federated machine learning model analysis system, a request that selects an analysis of one or more inference attacks to determine a presence of data of a subject in a training set of a federated machine learning model;
performing, by the federated machine learning model analysis system, the selected one or more inference attacks to determine the presence of the data of the subject in the training set of the federated machine learning model;
generating, by the federated machine learning model analysis system, respective success measurements for the selected one or more inference attacks based, at least in part, on the performance of the selected one or more inference attacks; and
providing, via the interface of the federated machine learning model analysis system, the respective success measurements for the selected one or more inference attacks.
9. The method of claim 8 , wherein one of the selected one or more inference attacks is a subject level membership inference attack that compares respective loss values determined from an application of the federated machine learning model to a subject distribution for the subject with a loss threshold to indicate the presence or absence of the subject distribution.
10. The method of claim 8 , wherein one of the selected one or more inference attacks is a subject level loss across rounds inference attack that determines respective loss values from an application of the federated machine learning model to a subject pool for the subject determined at different respective training rounds to indicate the presence or absence of the subject according to a trend analysis of the respective loss values.
11. The method of claim 8 , wherein one of the selected one or more inference attacks is a subject level loss across neighborhood inference attack that determines respective loss values from an application of the federated machine learning model to samples within a ball region of the subject with respectively added noise to indicate the presence or absence of the subject according to a comparison with a loss threshold.
12. The method of claim 8 , wherein the respective success measurements for the selected one or more inference attacks are provided as part of a result that also comprises one or more remedial actions.
13. The method of claim 12 , wherein the one or more remedial actions include a mitigation action to be applied when training the federated machine learning model.
14. The method of claim 8 , wherein the request that selects the one or more inference attacks further includes the subject and subject data to use for performing the selected one or more inference attacks.
15. One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices, cause the one or more computing devices to implement:
receiving, via an interface of a federated machine learning model analysis system, a request that selects an analysis of one or more inference attacks to determine a presence of data of a subject in a training set of a federated machine learning model;
performing, by the federated machine learning model analysis system, the selected one or more inference attacks to determine the presence of the data of the subject in the training set of the federated machine learning model;
analyzing respective inferences produced by the federated machine learning model as part of performing the selected one or more inference attacks to determine respective success measurements for the selected one or more inference attacks; and
providing, via the interface of the federated machine learning model analysis system, the respective success measurements for the selected one or more inference attacks.
16. The one or more non-transitory, computer-readable storage media of claim 15 , wherein one of the selected one or more inference attacks is a subject level membership inference attack that compares respective loss values determined from an application of the federated machine learning model to a subject distribution for the subject with a loss threshold to indicate the presence or absence of the subject distribution.
17. The one or more non-transitory, computer-readable storage media of claim 15 , wherein one of the selected one or more inference attacks is a subject level loss across rounds inference attack that determines respective loss values from an application of the federated machine learning model to a subject pool determined at different respective training rounds to indicate the presence or absence of the subject according to a trend analysis of the respective loss values.
18. The one or more non-transitory, computer-readable storage media of claim 15 , wherein one of the selected one or more inference attacks is a subject level loss across neighborhood inference attack that determines respective loss values from an application of the federated machine learning model to samples within a ball region of the subject with respectively added noise to indicate the presence or absence of the subject according to a comparison with a loss threshold.
19. The one or more non-transitory, computer-readable storage media of claim 15 , wherein the respective success measurements for the selected one or more inference attacks are provided as part of a result that also comprises one or more remedial actions.
20. The one or more non-transitory, computer-readable storage media of claim 19 , wherein the one or more remedial actions include a mitigation action to be applied when training the federated machine learning model.