Use Perturbations when Learning from Explanations

Summary

This paper proposes a novel approach to Machine Learning from Explanations (MLX), the setting in which data points for learning are paired with human-annotated “explanations”, usually in the form of image masks marking regions that are known to be irrelevant with respect to the downstream prediction task. Previous methods for MLX made use of regularization-based techniques, forcing the learnt function to be locally independent of the irrelevant features for the training data points. The authors show that these methods require strong model smoothing to be effective, to the extent that downstream task performance is heavily penalized. They then propose “robustness-based” methods, which force the model to be robust to perturbation across the irrelevant features, and show theoretically and practically that these techniques work even without heavy regularization smoothing. The authors then show, using three benchmarks, that their technique works in practice, but best results are obtained by combining robustness and regularization-based methods.

Strengths

The paper improves on methods from an existing problem setting, applies robustness techniques where they had not been tried before, and convincingly shows both with theory and experiments that such techniques work. The theorems, despite their simplified setting, intuitively demonstrate why the approach works better than the alternatives. The experimental results convincingly show that the robustness-based techniques work better than regularization-based techniques.

Weaknesses

While the contribution itself is nicely self-contained and perfectly valid when taken at face value (that in the MLX setting, training GPs or simple CNNs, robustness-based methods are better than regularization-based methods), one cannot help but wonder whether the entire setup is relevant or interesting, considering current state of the art architectures and how they attend to their input. Attention-based architectures like ViTs, due to intermediate-layer attention maps, lend themselves to very natural investigations commensurable to the kind of pixel-wise explanations used in this work. One can imagine that, as these transformer models also easily incorporate masking, the whole MLX problem could be reframed to be about either manipulating attention maps, or dynamically learning attention masks. And example is [Yang et al. (2023)](https://openaccess.thecvf.com/content/CVPR2023/html/Yang_Improving_Visual_Grounding_by_Encouraging_Consistent_Gradient-Based_Explanations_CVPR_2023_paper.html). Addressing these concerns either in the introduction or related work could help contextualize the paper’s contributions.

Questions

It would be interesting, to place the present work in perspective, to at least spend some time in related work to investigate how “explanations” are currently treated in the literature on transformer-based image models like ViTs. Is there any existing literature showing techniques for MLX (maybe not labeled as such) that are dependent on a particular architecture choice? If so, what makes the present work still interesting? Edit: Increased the score after author rebuttal addressing the points above - see comment below.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The authors briefly mention limitations related to the possibility, in future work, to learn from incomplete explanation data. It would be interesting to add a segment related to the points raised in Weaknesses and Questions.

Summary

This paper reinterprets Machine learning from explanations (MLX) as a robustness problem, where human explanations define a lower-dimensional manifold for perturbations. The paper points out that the previous regularization-based MLX approaches require strong model smoothing in order to be globally effective at reducing shortcut learning. The authors propose a novel approach that combines robust training methods with an earlier MLX technique, achieving state-of-the-art results on both synthetic and real-world benchmarks. The theoretical and empirical analyses explaining how the combination of robustness and regularization can reduce the need for strong model smoothing are provided.

Strengths

1. The paper is well-structured and clearly organized, making it easy to follow; 2. The authors offer theoretical analyses to explain how the combination of robustness and regularization can minimize the need for strong model smoothing, which adds to the rigor and solidity of the paper.

Weaknesses

1. The contribution of the proposed method is restricted to the combination of two established robust training techniques with an existing MLX approach; 2. Although the authors thoroughly review various regularization-based MLX approaches in the introduction, the paper only showcases the effectiveness of combining robust training methods with a single MLX approach, Grad-Reg. It remains unclear whether the proposed approach can be applied to other MLX methods, and further investigation is necessary to determine its generalizability. In addition, Table 1 does not include the results of the more recent MLX approaches that are well discussed in the introduction section.

Questions

1. Is it possible to apply the proposed methods to other MLX approaches? To better understand the generalizability of the approach, it would be helpful to see the performance of robust training methods combined with various MLX methods. 2. The authors mention in lines 277-279 that they evaluate in-domain test images with background pixels replaced by a constant pixel value. Have you tested the methods' performance under other out-of-distribution (OOD) scenarios, such as replacing background pixels with the background pixels from randomly selected images? 3. A sensitivity analysis on the hyper-parameters should be provided.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

See weaknesses. At this moment, the main limitations of the paper revolve around the limited novelty and generalizability of the proposed method. I tend to reject the paper primarily for this reason. However, I may reconsider my evaluation if the authors can provide solid evidence demonstrating the generalizability of their approach.

Summary

This paper proposes a new approach to machine learning from explanations (MLX). This new approach is still based on human-provided explanations of (ir)relevant features for each input in training, but recasts the MLX problem itself essentially into a robustness problem. The authors achieve SOTA performance when combining their method with previous ones on several benchmarks.

Strengths

The authors show that the need for strong parameter smoothing of earlier approaches can be overcome, and they achieve SOTA performance on several benchmarks. Their method is intuitive and easy to understand.

Weaknesses

Obtaining human-specified masks is at best a lot of effort and in many cases simply not available, which limits the scope of problems for which their method can be applied.

Questions

Very minor: What is the bolding criteria for Table 1? Usually, the best model is in bold, but here there are 1, 2, or 3 models bolded depending on the dataset, and it's not clear what determined the number of bolded models for each.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

It would be good to expand the "Limitations" section and provide more detail on, e.g., when the scaling breaks to large NNs, i.e. provide some guidance here about when one should expect the robustness methods to no longer be feasible to use.

Summary

The paper is in the domain of MLX (machine learning from explanations). In this approach, human annotated data for each input example is available denoting features that are _relevant_ and which are _irrelevant_. It is desired that the model doesn't learn from irrelevant features. In this paper, the authors utilize robustness for this domain. To elaborate, they expect the model to be robust to perturbations along the features which are considered irrelevant. According to the authors, this is the first use of this technique in the domain of MLX. They also combine with existing regularization-based approaches. A theoretical framework is provided and approach evaluated on three datasets -- Decoy-MNIST, Plant, ISIC -- which are designed to capture the extent to which the models are learning from the irrelevant features. Combination of the robustness-based approaches with regularization-based approaches is shown to outperform prior approaches.

Strengths

The use of robustness for MLX is well-motivated, and as per the paper, novel. The empirical results appear to back the utility of combining robustness with existing techniques. The paper is well-written.

Weaknesses

The prior works (such as Sagawa et al., 2019; Piratla et al., 2021) have several more evaluation datasets, which the approach is not evaluated on. The generality and robustness (pardon the pun) of the approach is not completely clear.

Questions

Questions - Please address the choice of limiting the evaluation to the 3 datasets used. Minor suggestion (no response needed) - L63: State the domain of m^(n) - Consider placing figure closer to their first reference. For example, Figure 2 is referenced on Pg 2 but appears on Pg 6. - L70: "while not exploiting" -- please provide a the definition for this at this stage (addendum: if there is one at all)

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

Yes

Reply to the Authors

We thank the authors very much for their very detailed and on-point reply. Specifically, I found the authors to have gone above and beyond in addressing the specific point I raised on architecture-dependent MLX technique. Their new experiment compares one such technique (SPAN) with theirs, and furthermore shows that combining the two improves performance, as predicted by their previous results. I would thus like to raise my score to a 7 (I do not however seem to find a way to edit my own review at the moment). Edit: I was now able to edit the review.