Survey - A Survey of Privacy Attacks in Machine Learning
- Privacy Preserving
- NN model should be a black box with only input and output
- attack could be at the model itself / acquire private training data from it
- the threat model
- 4 parts:
- different acess levels:just have the API / have full access to the model / training setting
- Attacks
- black-box atatcks: usually atack pretrained model service from the cloud
- white-box: have access to all model params / grads during training
- partial whitebox: make strong assumptions to black box but no full access
-
- attacks training / inference: attacks training means active attacker
- Taxnomy of threats:
- Membership Inference: whether input x is used as part of training set
- Reconstrcut Attack: recreate samples from their label using the model
- also called attribute inference / model inversion
- Property Inference Attacks: extract the dataset property which is not explicitly listed as label:
- e.g. find the ratio of women/man in patient dataset when the gender is not the label
- Model Extraction: black-box, want to acquire model
- sometimes wants to generate a substitiude model, replicate the decision boundary
- often wants to be efficient, as few queries as possible
- sometimes also predicting the attributes of the model, the hyper-pram of reg, the optimizer type, the activation type
- Attacks against centralized supervised learning
- membership attck(shadow training) - lies in that models behave differently when they see data does not belong to the trainig set
- reconstruction attack
- property inference attack
- Model Extraction attacks:
- view model param/hyper-parm in objective as the unknown
- for linear binary classifier with d dimension only needs d+1 queries, for non-linear perceptrons use optimization techniques as BFGS/SGD
- similar to Active Learning, which has an external oracle to provide label for inquery
- some others view it as finding the most valuable data to query:
- use data not synthectic but from other fomain
- unsupervised techniques as MixMatch
- membership attck(shadow training) - lies in that models behave differently when they see data does not belong to the trainig set
- Attacks against distributed learning
- federated learning setting, its not safe cause each client could access the model parameters
- federated learning setting, its not safe cause each client could access the model parameters
- Defenses
- Differentiabel Privacy:
- add gaussian / laplacian noise to the output as the \epsilon
- other approaches:
- regularization
- prediction vector tampering(干预)
- model compression
- ensembel
- noisy data
- weight quant
- selective sharing
- membership inference
