Exploring and Interacting with the Set of Good Sparse Generalized Additive Models

Summary

In contrast to the conventional machine learning paradigm, which typically yields a single model, Rashomon sets comprise a collection of near-optimal models. These sets allow users to select a model based on their specific preferences. This paper focuses on constructing Rashomon sets for sparse generalised additive models (GAMs). Furthermore, the paper demonstrates that by constructing Rashomon sets, users can estimate variable importance, enforce monotonic constraints, and interact with the shape functions of GAMs.

Strengths

The paper addresses a novel problem of constructing Rashomon sets for sparse generalized additive models (GAMs). It highlights the scarcity of prior work in this specific area, emphasizing the novelty and significance of the research.

Weaknesses

While the paper appears to be well-executed, I must admit that I lack familiarity with the concepts of Rashomon sets and generalized additive models. As a result, I am unable to provide an in-depth review or offer a confident assessment of the paper's claims. This limitation is due to my limited knowledge in these specific areas, despite my previous research experience in explainable AI.

Questions

I have difficulty understanding the distinction between fixed support sets and different support sets in Section 3.2. Could you please provide further clarification or explanation on this matter?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

I appreciate the solidity of the paper, although I must admit that I personally find it challenging to comprehend its content. I want to emphasize that this difficulty arises from my own limitations as a reviewer, rather than any shortcomings of the paper itself. As I am not familiar with the surrounding literature in this field, I can’t provide a judgment.

Summary

this paper designs a new editable gam model. They leverage the ellipsoid to approximate the optimal solution set. The model allows to edit the model according to different use-cases and requirements. The edited parameter could be solved by constrained quadratic problem. They experiment the model and edited use-cases at four datasets.

Strengths

1.providing a novel perspective of constructing editable GAM 2.proving the upper and lower bound of the variances' importance 3. showing the specific result of edited use-case of multi-datasets

Weaknesses

1. the procedure of approximating the roshomon set has high complexity. The experiments don't evaluate the overhead of this part. 2. though the quadratic problem is easy to solve by programming, the constraint of ellipsoid and high dimensions of features may make the procedure to be high complexity. It needs to evalute the overhead in the experiment. Besides, as Q has large dimension, it is expensive to store it.

Questions

1, how many bins do you used in EBM? 2, how to use the proposed method when the final model is the bagging of multiple GAMs？

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

yes

Summary

This paper presents a novel algorithm (and an additional variant) to learn the Rashomon set of sparse generalized additive models (GAMs) approximated by an ellipsoid in the parameter space. The paper also presents 4 interesting use cases of the proposed algorithm. It includes a rich experiments section. In the experiment section, the authors compared the proposed algorithm with a few others and showed that the proposed algorithm has very favorable results in terms of both precision and volume. The experiment section also includes the demonstrations of the proposed algorithm in several use cases.

Strengths

Interesting problem, rigorous formulation, novel algorithm, clear presentation, and favorable experiment results.

Weaknesses

The problem this papers studies is a niche in the general ML problem space.

Questions

1. line 89, p. 3: You mentioned "$\omega$ also includes $\omega_0$". I assume this is just to say that the notation $\omega$ will include the bias term $\omega_0$. However, we usually don't apply $L_2$-regularization to the bias term, so regularization weight $\pi_0$ for the bias term still remains zero, right?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

n/a