Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable

Summary

This paper studies reconstruction attacks on machine unlearning. The authors propose a reconstruction attack that can accurately recovers the deleted sample given the pair of linear models before and after sample deletion. This is made possible by leveraging the closed-form single-sample training algorithm for linear regression as well as the ability to accurately estimate the covariance matrix of the training data from a public dataset. They also extend the attack to the setting where the model consists of a (fixed and known) embedding function, followed by a trained linear layer.

Strengths

- The problem studied in this paper is interesting. - The proposed attack achieves good performance for linear regression models.

Weaknesses

- The proposed attack is based on strong assumptions. This paper assumes that the attacker has knowledge of the training data distribution, the model’s loss function, and the embedding function. However, this information is usually not published by the model maintainer. It is unclear how the attacker can obtain this information in practice. The authors should provide more details to justify these assumptions. - The proposed attack has limited applications. This paper mainly focuses on linear models, and it is unclear whether the proposed attack can maintain good performance with more complex models. - It is unclear which unlearning method is adopted in the experiments. Will the attack performance vary when different unlearning methods are applied? - The proposed attack relies on a public dataset with the same distribution as the training data. The authors do not provide information about the size of this dataset in their experiments. It would be more convincing if they could evaluate the effect of dataset size on the attack performance.

Questions

See above weaknesses.

Rating

4

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

Yes

Summary

This paper presents reconstruction attacks against Machine Unlearning in the following sense: the attacker is assumed to have access to a model's parameters before and after the removal of a single data point; they then produce a guess for this point, which is evaluated in terms of its cosine similarity to the original data point.

Strengths

This paper is nice and easy to read. The description of the attacks is easy to follow, and the theoretical derivations are interesting. The specific application of reconstruction attacks against machine unlearning is, to my knowledge, novel; it stems from a large body studying the privacy of machine unlearning. The presented problem is also well-scoped, and this research opens the space for new studies in the area.

Weaknesses

1. The threat model is unrealistic: it is far fetched to assume that the attacker has access to the model parameters (before and after unlearning), and yet at the same time to assume that they cannot see the target point x in this process. Now, strong assumptions such as this one have been used in prior literature; but usually their purpose is to set upper bounds on the adversary when proving the security of defenses. That is not the case here. A second assumption that is quite strange is that the attacker somehow knows the model's parameters, yet they don't know the training set Xpriv, and they need to rely on a public one. 2. The authors focus on the very limited and quite simplistic scenario specified above. Yet they had various options for exploration: - what if the attacker only has black-box access to the model? Based on similar prior work that evaluated both white- and black-box access (e.g., Balle et al.), I would expect your attacks to transfer well. - You mentioned DP in several places, yet provided no evaluation of said defense: what parameter set can prevent these attacks? - What if more than 1 points were unlearned at once? Would your attack apply?

Questions

Can you please explain your threat model choices (see Weaknesses above)? Typo: "To simply the notation" Baseline: an interesting baseline to consider would be the point from the public set that is closest to the target point in the private set; this would intuitively be a better baseline than MaxDiff. Could your methods beat this baseline?

Rating

6

Confidence

5

Soundness

4

Presentation

4

Contribution

2

Limitations

These were appropriately discussed, although the authors should better emphasize that the threat model is not realistic.

Summary

The authors propose an attack that can accurately recover unlearned samples through reconstruction attacks on linear regression models. They extend this work to include linear models with fixed embeddings and generalize it to more generic loss functions and model architectures by employing Newton’s method for the reconstruction attack. This work significantly contributes to understanding the privacy vulnerabilities in machine unlearning.

Strengths

1. The authors provide rigorous theoretical proof of the reconstruction attack. 2. They conducted thorough experiments across different tasks, datasets, and architectures, demonstrating the effectiveness of their attack. 3. The extension of the work to linear models with fixed embeddings and the generalization to other loss functions and model architectures showcase the adaptability and robustness of their method.

Weaknesses

1. This work is limited to the exact unlearning scenario, i.e. retraining from scratch without the unlearned data, and focuses solely on unlearning a single data point. In contrast, the scenarios that have received more attention in unlearning research involve approximate unlearning and unlearning multiple data points at the same time. 2. The experimental evaluation lacks diversity in metrics, which could provide a more comprehensive understanding of the attack's effectiveness. 3. There is limited discussion on the potential defenses against the proposed attack.

Questions

1. Is “Avg” a valid or commonly used baseline? It seems too straightforward to predict the deleted example as an average of the public samples, as described in Section 4. Can the authors elaborate on the principle or intuition behind this choice? 2. How robust is this attack across different configurations? For example, how does the attack's performance vary when using different model architectures on the same dataset? 3. Can the authors elaborate on the computational complexity and scalability of the proposed attack method when applied to various datasets and model architectures? 4. What practical countermeasures can be implemented to mitigate the identified privacy vulnerabilities (e.g. the proposed reconstruction attack in this paper) in machine unlearning?

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

See Weaknesses Section.

Summary

This work focuses on investigating privacy issues in machine unlearning. Specifically, assuming the availability of model parameters before and after unlearning, as well as the ability to sample data from the original data distribution, the proposed reconstruction attack aims to recover deleted samples. By analyzing the training objective of linear regression, the study found that the difference between the parameters before and after unlearning is proportional to the deleted sample. Based on this observation, this study proposes an algorithm to extract deleted samples accurately. The method extends to more complex models and arbitrary loss functions using Newton's method to approximate the parameter update process.

Strengths

1. The topic of the study, concerning privacy risks in unlearning, is crucial. Since the data deleted in unlearning usually has high privacy sensitivity, recovering such data poses a significant threat. 2. The proposed algorithm is elegant and achieves near-perfect results in linear regression. It also has the potential to extend to more complex models.

Weaknesses

1. The assumptions are too strong. The authors assume access to model parameters before and after unlearning and the ability to sample from the original data distribution. The authors need to clarify under what circumstances an attacker could have the assumptions mentioned in the paper, especially the sampling ability, since the deleted data is typically highly sensitive or inappropriate, making sampling difficult. 2. The goal of this paper is to explore the privacy risks in machine unlearning. In real-world scenarios, machine unlearning is achieved through existing unlearning methods. However, in this paper, unlearning is implemented by retraining after removing the samples to be unlearned. This creates a gap between the resulting model and the model obtained through actual unlearning methods. Therefore, it is questionable that the experimental results obtained from such a model can directly demonstrate the privacy risks associated with machine unlearning in real-world situations.

Questions

1. In what scenarios is the threat model proposed in the paper realistic? 2. Is the proposed method effective when unlearning is performed using existing unlearning methods?

Rating

5

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

None