Learning with Explanation Constraints

Summary

This paper tries to answer an important and fundamental question: why can interpretability constraints on model parameters improve model performance. The authors developed an analytic framework. They first define constraints as a functional on a model function and a data point, and the model satisfying the constraint is hence represented as the functional value lies in a set. Following this setup, they defined the surrogate loss based on the constraint violation, and showed the generalization bound on this constraint violation loss. They showed that the constraints helped shrink the hypothesis class. They also showed that with gradient constraints, linear and 2-layer NN models could benefit in requiring fewer labeled data. They also discussed optimization algorithms through projections that could make the model satisfy the constraints.

Strengths

This paper tries to answer an important and fundamental question: why can interpretability constraints on model parameters improve model performance. The presentation of this paper is clear, techniques sound, and contributions good.

Weaknesses

The paper lacks a discussion on the stringency of their conditions in theorems, which makes it more difficult to judge the significance of the contribution. The paper focus on gradient constraints and lacks a discussion and related work on model shape constraints, such as monotonicity and convexity.

Questions

Please address the comments in the weakness section.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

This paper doesn't have potential negative societal impact. The authors are encouraged to discuss the plausibility of the conditions of their theorems.

Summary

Traditionally, learning from labelled examples has been well studied. However, with the advent of large models (and few-shot learning) and risk of misspecifications (spurious correlations), need for augmenting supervision such as with example explanations is gaining wider interest. This paper deals with this timely topic of analysing when and how learning from explanations can improve sample efficiency. Although learning from explanations is also promising in reducing systematic biases, this paper focusses on sample efficiency. They also propose an interesting learning algorithm to accommodate explanation constraints. I believe the topic and analysis is timely but I have a few concenrns regarding evaluation and analysis.

Strengths

- Important practical problem. - Well written and easy to follow. - The proposed variational method is smart, which can exploit both labelled, unlabelled data along with explanations.

Weaknesses

Disclaimer: I have not checked the proofs. But the statements seem correct. Consider the following also as questions. *How are explanations different?* Fundamentally, I did not understand why explanation constraints are given a status different from example labels. Can we not simply consider them to be additional labels and minimize the joint risk such as in the Lagrangian method or Rieger et.al. and handle the error bound using the classical analysis as shown between L161 and L162? Especially since the explanations are imposed only approximately through objective functions. *Clarification regarding Theorem 2.2.* From L 156: "... help with our learning by shrinking the hypothesis class $H$ to $H_{\phi,\tau+\epsilon_k}$". Although this sounds intuitively correct, from the definition of $H$, isn't $H_{\phi, \tau+\epsilon_k}$ larger than $H$? Moreover, I do not see how constraints help improve the sample complexity since both $R_n(H_{\phi,\tau+\epsilon_k})$ and $err_D(h^*_{\tau-\epsilon_k})$ increase for a non-zero value of $\epsilon_k$? *Motivation for variational method.* The need for variational method is motivated to avoid evaluating gradients of a potentially complicated surrogate loss: $\phi$, however even the variational method needs to evaluate the same when optimizing for $f$ in the first step. *Evaluation concern*. Lagrangian method may have performed poorly because it is not exposed to unlabelled data unlike variational. What happens if we add an additional term to lagrangian objective on unlabelled data that is similar to self-training? Otherwise, I do not see why lagrangian may poorly perform when compared with variational at least when we can compute gradient of gradients well (which is not much of a problem these days). *Real world performance gains are unimpressive*. (Section 5.2 and Figure 5). I cannot clearly see the observation noted in L312-L313: "We observe that our variational objective achieves better performance than all other baseline approaches, across varying amounts of labeled data." The weak labellers and the task should be better defined. How are the gradients (of $\phi$) computed (for say largrangian method) when weak learners are used for imposing constraints?

Questions

Please see weaknesses above.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

Addressed.

Summary

In this work, the authors theoretically investigate learning with constraints on the gradient of the model. Like other works in a similar vein, this paper focuses on gradient-based explanations and on neural networks. Unlike related works, however, this paper formulates explanation constraints as being over the data distribution rather than locally to a single input. In this setting, the authors key contributions come from their focus on a learning theoretic analysis of models that satisfy their constraints (called EPAC models) which I find to be significant contribution. This paper is timely given the current focus on trustworthy AI and an emphasis on explainable AI in proposed legislation. I feel the analysis is simplistic in some places. Additionally, it is unused in the experimental section, and a few key prior works are not mentioned. I do believe this work provides a solid learning theoretic foundation for the budding sub-field of machine learning from explanations (MLX) and for works studying different kinds of learning constraints (e.g., adversarial certification). I will note that though I have given the paper a borderline score, I will happily update my score to an accept if the authors can substantially address my comments during the rebuttal period.

Strengths

The paper tackles a considerably interesting problem that will certainly have impact on future works and can serve as an important basis for understanding a budding sub-field of machine learning research. The formulations all appear sound and correct to the best of my knowledge. The paper is very easy to follow as it is well written. I raise a few minor points about the writing below, but in general I appreciate the high-quality of the paper’s presentation.

Weaknesses

The main drawback of the paper is that the theory and experiments are almost wholly disjointed. The first portion of the paper focuses specifically on theoretical guarantees using standard learning theoretic tools; however, I do note see where any of these tools are experimentally evaluated even in the toy experiments. Moreover, machine learning from explanation (MLX) papers and even robust explanation constraint papers focus on datasets and models that are considerably more sophisticated than what is proposed in this paper. * EPAC-ERM objective is exactly the same as what is proposed in prior machine learning from explanation (MLX) works, but no discussion is given. * For completeness the work should cite and at least discuss if not experimentally compare to papers [1][2] and [3] There are several scientifically vague sentences which I find inappropriate without further discussion of formalization. Three in particular come to mind: Another part of the paper I find lacking is the existence of several vague scientific statements that are not backed up by formal argument or good intuition. I give three examples and my issues below: “We argue that in many cases, n labeled data are the most expensive to annotate. The k data points with explanations also have non-trivial cost; they require an expert to provide the annotated explanation or provide a surrogate loss.” - Page 6. It seems that for images at least, eliciting explanation annotations from humans is considerably more costly. Moreover, the authors claim that they “argue” the case of having a loss function, the k data points with explanations would be less costly, but I see no such argument. It is not obvious to me that eliciting a high-quality loss function $\phi$ would be any less expensive than eliciting annotations themselves. Happy to hear an argument to the contrary, but on its own this statement feels unscientific and needs to be buttressed with an actual argument. To this end, I would find it very interesting, and I think it would also be interesting to the community working on these problems if a query/sample complexity was explicitly modeled and compared (i.e., the sample complexity of learning a loss function versus the sample complexity of learning a EPAC model from concretely provided labels) “Computing the gradients of this surrogate loss  in turn is much more expensive compared to gradients of the empirical risk.“ - Page 7. Again, I feel this is a vague and unjustified statement. The gradient of the surrogate loss for explanations is of course more expensive than the standard classification loss, but in previous papers regularization of explanations based on gradients has been done for even ResNet-50 sized models without any hint of computational strain. So, while the general case of the loss might be considerably more difficult to differentiate, in the case of studying gradient-based explanations and their losses dont seem “much more expensive” “Thus, the decoupled approach to solving the EPAC-ERM objective is in general more computationally convenient. “ - Page 7. I feel this statement would be easy enough to back up experimentally in the experimental setting put forward in this work. One can simply provide the numbers showing this. However, I am curious if this remains the case when the method is scaled up. 
 [1] Ross et. Al., 2017 https://arxiv.org/abs/1703.03717 (this one is cited but not compared experimentally which I think it ought to be) [2] Wicker et. Al. 2022 https://arxiv.org/abs/2212.08507 (this ought to be cited, but is in a different vein so no experimental comparison needed) [3] Czarnecki et. Al. 2017 https://arxiv.org/pdf/1706.04859.pdf (Similar aim, ought to be discussed again perhaps not experimentally compared)

Questions

See the above section.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

See the weaknesses section

Summary

Disclaimer: I am not an expert in Learning Theory, but I come from a background of interpretability and explanation model’s decision making. So I am familiar with the methods that this work is trying to describe and some related works. I will review this paper modestly, and hope authors and AC can understand. In the papers, the authors studied recent methods in regularization of prediction models using explanations. An example of explanation constraints would be enforcing certain properties of the gradients of the objective w.r.t. to the data. Another would be requiring a separate teacher model that takes in the prediction model and data and require the teacher model to have specific desired outputs. Authors analyze these explanation constraints from a statistical learning perspective, such as presenting two different explanation constraints and the analysis of their learnability and generalization bounds. Then, the authors proposed the algorithm for learning with explanation constraints, and a variational method for solving the objective. Experiments performed on synthetic data and simple real-world data demonstrates that their variational approach achieves better performance than standard approach.

Strengths

The paper is overall well-organized and well-written. The notation seems clear and not cluttered. The formulation and definitions are also not difficult to follow, and the writing describes most of the motivation of formulations well. Since I am not an expert, I cannot comment on the novelty of the work. Nonetheless theoretical analysis of explanation methods or methods that incorporate explanations into training are rare. So this work could potentially be a start to introducing theory in explanation methods. The authors have cited sufficient related works in the appendix. The experiments in this work are solid, where authors compared with multiple baselines and sufficiently discussed the results. The setup of the experiments also seem fair and complete. Appendix also supplies a range of ablation study such as hyperparameters.

Weaknesses

I do not observe any major weaknesses in this work. Although, in terms of writing, the transition into section 4.1 was a bit of a jump. It was not immediately clear to me why there is a need for variational method. Although in experiments it shows to obtain better performance that non-variational approaches, it wasn’t immediately obvious what prompted the authors to develop a variational approach, hence there seems to be a gap between going from Line 234 to line 236.

Questions

1. The authors claim that in Line 29 “An attractive facet of the latter is that we can automatically generate model-based explanations given unlabeled data points.” Can the authors cite works that generate model-based explanations given unlabeled data or clarify what this means?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors did not address any limitation of their work, but addressed its social impact. One limitation could be that in this work, authors seem to focus on restricting gradients as a explanation constraints. It will be great if future work can also address other versions of explanation constraints, and potential to extensions such as feature selection methods as explanation methods. Analysis of models beyond two-layer neural network regime could potentially be discussed.

Summary

This paper presents the concept of learning from explanations and theoretically shows that learning from explanations can improve the standard excess risk bounds in the agnostic setting. They treat explanations as constraints in the ERM procedure and introduce an extension of PAC model referred to as EPAC model. The paper introduces formal notions of explanations, EPAC learnability, explanation constraint sets etc. The study analyzes the impact of explanations on model learning using standard learning theoretic tools and characterizes the constraints for linear models and two-layer neural networks. The statistical advantages of explanations stem from their role in constraining the hypothesis class: explanation samples improve the estimation of the population explanation constraint, thereby further restricting the hypothesis class. Further, the paper provides an algorithmic solution based on variational approximation to solve the problem of learning with explanation constraints. The proposed solution works better than other methods like augmented Lagrangian.

Strengths

Strengths: ------------- 1. The paper provides a theoretical framework for studying learning with explanations by formalizing various key notions like explanation functionals, explanation constraint set, EPAC model and EPAC learnability etc. 2. Their analysis provides insights into why learning with explanations could improve the overall excess error of the ERM solution. The key insight is that having explanation constraints in the ERM procedure effectively reduces the size of the model class leading to smaller excess risk upper bounds. 3. The paper also provides an algorithm solution to solve the ERM objective while satisfying the explanation constraints and the empirical results show that the proposed method works better than augmented Lagrangian and other baselines.

Weaknesses

Weaknesses and Questions: --------------------------------- 1. Overall I like the ideas presented in the paper but it was a bit tedious to get exactly what is being done. It first appeared that the paper is proposing a method to learn the explanation functionals i.e. learning a model that can generate explanations by learning from some labeled explanation data. However, the goal is to study how using the explanation information improves the model performance. It would be helpful to improve the presentation to make it clear early. 2. I am not convinced about the notion of explanations introduced. It is assumed that explanations are some vectors. Shouldn’t explanations be human-readable and be in natural language? Then how do you pose the constraint sets in natural language? I might be missing something here as I have not followed explainability research. 3. I didn't quite get why explanation constraints are special? In general, one could put some constraints (e.g. regularization, etc.) along with the ERM objective to reduce the size of the hypothesis class, which will lead to a reduction in the upper bound of the excess risk. Could you please provide some justification or motivation for why the explanation constraints are special? 4. Figure 2. could be improved with some annotations and markers instead of just colors alone. The figures in experiments might be better when run with more random seeds.

Questions

Please see above.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Yes.

Response to author rebuttal

Dear authors, Thank you for responding to my questions. I think my concerns are not adequately addressed. My point 3 was to get better understanding of the theoretical results. They seem to be valid for any constrained ERM. How do we interpret them in the context of explanation constraints? I am looking for a bit more nuanced understanding of these results from explanations point of view. I am also not convinced about why these results are novel or interesting. In general one could put constraints on ERM objective (e.g. regularization etc.) and that will lead to reduction of hypothesis class and hence smaller upper bounds. In these cases the results are more refined, providing more insights on constraint parameters and how they affect the bounds. In this regard, I don't find Theorem 2.2 satisfactory. Do you have any lower bounds on the generalization errors to claim that learning with explanations is helpful? The current conclusions are based on comparing two upper bounds on the generalization errors. In my view, the presentation could be improved to make it easier to follow. Here are some of my suggestions, a. Please include some intuitive examples of explanation constraints early in the paper. b. May be have a few figures, showing these constraints and how they affect the learning procedure. c. Provide a brief summary of the explanation constraints considered in the literature and why they can be abstracted out mathematically the way it is done in the paper.

Response

I thank the authors for their response. The authors have sufficiently addressed my questions and concerns. As of now, I do not have further questions.

Further clarification

Thanks for the response, some of my concerns are now resolved. Points 2 and 4 are now resolved, thanks. I still have many other concerns. _Theoretical analysis_: I am not yet convinced why explanation constraints are special. This issue was also raised by Reviewer 3fR6, but not yet resolved. Why not treat explanation constraints like any other constraints? It is not hard to see that constraints provide additional supervision, which would improve label efficiency. On the other extreme, supervision on explanations is not that different from label supervision. To elaborate on this, if the underlying hypothesis class is drawn from a GP prior, then observations on function gradient are readily accommodated just as observations on function values [1]. More crisply, what is the significance of the theoretic results in relation to explanations beyond just saying that additional supervision through constraints improved label efficiency. _Experiments_ Q: Could you please clarify how $\lambda$ is obtained in the Augmented Lagrangian objective? Was $\lambda$ fixed? Was it obtained through alternate max-min steps typical of Lagrangian optimization? Q: Variational vs Lagrangian. The advantage of variational over Lagrangian is argued to be only computational. Figure 4 needs to be justified as to why Variational is satisfying explanation constraints better than Lagrangian. Few other issues: Datasets are non-standard. Rieger et.al. and Ross et.al. contain many other standard datasets like Decoy-MNIST or ISIC dataset. But the datasets used to report in this paper have not been used before for this problem to the best of my knowledge. The reason for choosing not to evaluate on standard datasets needs to be justified. Moreover, evaluation needs to be more thorough to demonstrate the generality to new explanations (like the one used in Rieger et.al.), which is also a concern shared by Reviewer o4F1. Moreover, I agree with Reviewer o4F1 that experiment and analysis are almost disjoint. Although I agree that the experiments corroborate label efficiency claim from theory, the connection between analysis and experiments is still very flimsy. For instance, theory did not explain why variational is better than Lagrangian or why Lagrangian failed to satisfy explanation constraints as shown in Fig. 4 (right) or explain the trend in rate of reduction of loss with number of training examples. I am updating my score accordingly. [1] Hennig, P., Osborne, M. A., and Kersting, H. P. Probabilistic Numerics: Computation as Machine Learning. Cambridge University Press, 2022. doi: 10.1017/9781316681411. (See chapter 4)

Thank you for the clarifications!

I would like to thank the authors for their very detailed response. I found it quite convincing. After reading the response and re-reading my review, I think my point about the experimental section being disjoint from the theory was too harsh. Moreover, I have had time to more carefully examine the Appendices which I found interesting and comprehensive. I have increased my score with the assumption that in the final version the authors properly cite the MLX/explanation constraint literature and make the changes they say they will. Thanks again :)