The Contextual Lasso: Sparse Linear Models via Deep Neural Networks

Sparse linear models are one of several core tools for interpretable machine learning, a field of emerging importance as predictive models permeate decision-making in many domains. Unfortunately, sparse linear models are far less flexible as functions of their input features than black-box models like deep neural networks. With this capability gap in mind, we study a not-uncommon situation where the input features dichotomize into two groups: explanatory features, which are candidates for inclusion as variables in an interpretable model, and contextual features, which select from the candidate variables and determine their effects. This dichotomy leads us to the contextual lasso, a new statistical estimator that fits a sparse linear model to the explanatory features such that the sparsity pattern and coefficients vary as a function of the contextual features. The fitting process learns this function nonparametrically via a deep neural network. To attain sparse coefficients, we train the network with a novel lasso regularizer in the form of a projection layer that maps the network's output onto the space of $\ell_1$-constrained linear models. An extensive suite of experiments on real and synthetic data suggests that the learned models, which remain highly transparent, can be sparser than the regular lasso without sacrificing the predictive power of a standard deep neural network.

Paper

References (58)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer kkJW4/10 · confidence 3/52023-07-01

Summary

This paper proposes the contextual lasso, considering the contextual sparsity which allows feature relevance to depend on the prediction context.To solve the optimization problem of contextual lasso, deep neural networks is utilized to learn interpretable sparse linear models. Besides, an extensive experimental analysis of the new estimator illustrates its good prediction, interpretation, and selection properties in various settings.

Strengths

1. The main novelty of this paper is that it simultaneously considers the explanatory feature and contextual feature in the classical lasso framework. In this case, the classical lasso can be viewed as a special case of contextual lasso. 2. The performance compared to lasso shows that combining the contextual feature and explanatory feature as proposed in the paper leads to performance benefits. 3. The paper is well-written and technically sound.

Weaknesses

1. An essential problem of this paper is that how to accurately define or distinct the explanatory feature and contextual feature still remains unclear. The authors should offer more definition or evidence to explain the contextual feature. 2. The comparison methods are insufficient. Especially, since contextual explanation networks is a cousin of the contextual lasso, it is necessary to show the superiority of contextual lasso against contextual explanation networks (Al-Shedivat et al.,2020). 3. Though some experiments can validate the effectiveness of contextual lasso over lasso, it also lacks a time complexity analysis. 4. Theoretical analysis (e.g., generalization, optimization) seems weak for the proposed approach.

Questions

1. The authors should provide detailed explanation or definition about the contextual feature. Moreover, how to achieve the additional contextual feature from the real-world data should also be included. 2. The authors should add more comparison methods including the contextual explanation networks. 3. The authors should offer the time complexity analysis of contextual lasso. 4. At least, some discussions are necessary for the theoretical properties of contextual lasso, e.g., generalization ability compared with the previous Lasso-type methods.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

One of the limitations of contextual lasso referred by authors is that it can result in bias of the linear model coefficients towards zero. This drawback is also due to the L1-norm and indeed belongs to all lasso estimators. The authors propose a possible relaxing approach.

Reviewer DGcD6/10 · confidence 2/52023-07-04

Summary

This paper proposed the contextual lasso model to combine the interpretability of linear model and the express power of deep neural network. The input feature are split into explanatory features and contextual features. The explanatory features are used as input features for a generalized linear model, and the contextual features are feed into a neural network with a projection layer on top to compute the parameters of the linear model. The projection layer are formed as a constrained optimization to force the linear model parameters lie in a $L_1$ ball. Grouped features and other extensions are also discussed. Experiment results show the proposed model outperform traditional DNN model and contextual linear model without LASSO sparsity.

Strengths

The paper is well written and easy to follow. Using contextual features to compute parameters for an interpretable sparse linear model seems to be an interesting and novel idea. Many extensions are discussed. The empirical results also support that the proposed model can select sparse model while maintain better prediction performance.

Weaknesses

A straightforward idea for a sparse DNN model is to include all explanatory features and contextual features as input to a DNN, train the model with some regularizations, after some thresholding, check the weights in the first layer(or multiplication of indicator matrix for all layers) to decide whether a variable is select or not. With appropriate regularization, I think this can also select relevant variables, and the variable importance can be measured by the gradient to the variable. Conceptually, why does the contextual model better? For real data, how should we decide which features are explanatory features or contextual features? And empirically, It was a bit surprise for me that the baseline Lassonet always keep a relatively dense model, is it due to some poor hyper-parameter choice? please see the Questions section for some clarification.

Questions

1. The projection layer works on the average coefficient vector, how does it work with mini-batch training for DNN, does it just look at the coefficient in that mini-batch? 2. In the experiments, for DNN type model, does contextual features included in the input? In line 280, the DNN and Bassinet are applied to the original features, it might be beneficial to include results of DNN or lassonet with all spline features and contextual features as input 3. Looking at the results of lassonet, it almost select all variables for Parkinson and Energy data sets and select all variables when n gets large for simulation data. How is the regularization parameter determined for lassonet. If we chose a large regularization parameter manually to force the model to have similar sparsity level as the contextual lasso, would the prediction performance be much worse?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Reviewer mL1j5/10 · confidence 4/52023-07-06

Summary

This paper considers the predictive problem setting in which the input features are dichotomized into two groups: explanatory features and contextual features. For such a setting, the authors propose the contextual lasso, a statistical estimator that leverages the expressiveness of deep neural networks to learn interpretable sparse linear models with sparsity patterns that can vary based on the contextual features. More precisely, the authors proposed using a neural network to model the nonlinear effects of the contextual features on the explanatory features. In addition, to enforce sparsity among the coefficients for the explanatory features, the network is trained with a regularizer in the form of a projection layer that maps the output (dense) to sparse coefficients by performing a projection onto the $\ell_1$-ball. Experiments on real and synthetic data suggest that the contextual lasso leads to models which remain interpretable and can be sparser than the regular lasso, without sacrificing the predictive accuracy of standard deep neural networks.

Strengths

This work appears to be the first to propose a sparsity-inducing projection layer that maps a dense output of a neural network onto a space of $\ell_1$-constrained linear models by solving a constrained quadratic program. The benefit from the proposed estimator is that it can utilize the expressive power of deep neural networks and yet learn interpretable sparse linear models by encouraging contextual sparsity through an additional projection layer; all while allowing feature relevance to depend on the prediction context as opposed to determining a fixed set of relevant features. In addition to the classical contextual sparse learning setting, a variant of the sparsity-encouraging projection layer is also devised for the group lasso setting. The proposed estimator and the formulation of the quadratic program for determining the sparse coefficients are technically sound and well-supported. Moreover, the particular sparse learning setting that is being considered is well-supported by examples where the input features are inherently dichotomized into explanatory features and contextual features, and the need for contextual sparsity in such a setting is well-justified. The effectiveness of the contextual lasso has been evaluated on both synthetic and real data (in the standard setting and a setting with grouped explanatory features). The findings seem to suggest that, in contrast to other sparse linear models and a conventional DNN, which use either all or almost all available explanatory features, contextual lasso leads to linear models that are much simpler and involve less explanatory features on average. The paper is rather well written and organized. The notation is very clear, easy to follow, unambiguous and consistent throughout the paper.

Weaknesses

The problem considered in this work is a rather general one, as many applications involve a certain context that can be leveraged to select candidate features and determine their effects. Nevertheless, this generality of the problem is not matched by the extensiveness of the conducted experiments. It seems that the contribution’s potential significance is limited by (1) the narrow settings of the experimental design (particularly in the synthetic data generation process) and (2) the consideration of only two real-world applications given the pervasiveness of sparse learning problems surrounded by a certain prediction context. I would encourage the authors to consider additional datasets characterized by both explanatory and contextual features. Although this work is aimed at capturing the prediction context by leveraging the expressive power of neural networks to map contextual features to a sparse coefficient vector, there are other works that take different approaches to the same end. For instance, [1] proposed adaptive optimizers for structured sparsity for deep learning, but also discusses unstructured sparsity ($\ell_1$) which is directly relevant to this work. There are also other works, such as [2], that take a non-neural based approach to capturing prediction context but I believe are still relevant. Namely, [2] addresses the house pricing problem (provided as an example in Section 1 of this work) using a generalization of the group lasso to a graph setting, where the graph represents the context defined by the distances between the house locations. Finally, there are also approaches that are not directly applicable to the setting considered in this paper, however, in a more general sense, they are relevant to inducing sparsity in neural networks and should be considered as a part of the related work (e.g., [3]). [1] Deleu, T., & Bengio, Y. (2021). Structured sparsity inducing adaptive optimizers for deep learning. arXiv preprint arXiv:2102.03869. [2] Hallac, D., Leskovec, J., & Boyd, S. (2015, August). Network lasso: Clustering and optimization in large graphs. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 387-396). [3] Wang, P., He, X., Li, G., Zhao, T., & Cheng, J. (2020, April). Sparsity-inducing binarized neural networks. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 07, pp. 12192-12199). As previously suggested, other sparse linear estimator variants have been proposed to utilize contextual information for the same or similar setting as the one considered in this work. Such models include Network Lasso (mentioned in the previous comment), Graphical Lasso and its associated extensions, among others. In that regard, I am wondering why the authors did not consider including such estimators as additional baselines. Moreover, since this work proposes a separate projection layer to induce sparsity in the output of a neural network, I also believe that a comparison to a conventional neural network with a dropout layer applied on its outputs ($\eta$ values) would be appropriate.

Questions

My questions/suggestions are provided together with the weaknesses of the paper (where applicable and necessary).

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

A separate “Limitations” section has not been included in this paper. No potential negative societal impact has been declared in the main manuscript. For my improvement suggestions, please refer to the “Weaknesses” section of this review.

Reviewer BSpt5/10 · confidence 4/52023-07-07

Summary

In this paper, the authors present the contextual lasso, a new estimator that fits a sparse linear model on the explanatory features, allowing for the coefficients to vary as a function of the contextual features. This estimator generalizes the original lasso and the varying-coefficient model. The authors introduce a projection layer for efficient model training and furthermore, they develop a group version of the contextual lasso, which is analogous to the extension of lasso to the group lasso.

Strengths

The paper is well-written and structured. The contextual lasso has the potential to provide substantial advantages for interpretation over neural networks, while leveraging the expressive power of these neural networks through the coefficient functions. The experiments conducted on property sales data (depicted in Figure 1), as well as real data tested in Section 4, are convincing to see the interpretability of the contextual lasso without compromising its predictive accuracy.

Weaknesses

Although the method has not been explored in previous studies, it appears to be a relatively direct extension of the lasso and the varying-coefficient model. Besides the architecture of the projection layer for model training, the remaining parts of the paper, such as the extension to the group version, side constraints, pathwise optimization, and relaxed fit, are all straightforward extension from existing lasso literature. The synthetic data experiments seem somewhat artificial. It's unclear whether these experiments provide any numerical evidence supporting the benefits of the contextual lasso. Given that the data is generated from the same model as the contextual lasso, it's expected that the method which leverages the true structure of the data would outperform or perform comparable to other methods. Furthermore, using the proportion of nonzero features as an interpretability metric is not appropriate, as interpretability is much broader than just sparsity. It would be more convincing to readers if the authors re-evaluate this experiment and supplement it with additional experiments using real data, for instance, in classification scenarios.

Questions

Can the authors provide additional details regarding the experimental results on property sales shown in Figure 1? Merely displaying the fitted coefficients does not clearly indicate the reliability of the results. Are the model fits sufficiently accurate to validate that the interpretation of the coefficient functions is meaningful? Without further context of data, it's difficult to understand if the learned model accurately represents the true structure of the data. Introducing the projection layer presents an issue with the non-differentiability of the objective function. How does this impact the actual training of the model? For example, is the learning process consistently stable across multiple random initializations? In Section 4, the contextual lasso (or contextual group lasso) seems to outperform deep neural networks, but it's unclear whether the comparison is fair. The performance of deep neural networks can exceed the contextual lasso, depending on the chosen architecture. If the sample size is sufficient, deep neural networks should potentially offer superior performance compared to the contextual lasso, and depending on the applications, some practioners may favor the use of deep neural networks at the cost of sacrificing interpretability. It's important to make this clear in the section; currently, it reads that the contextual lasso surpasses deep neural networks in both predictive accuracy and interpretability. In the first line of introduction, it might be more appropriate to revise the statement "Sparse linear models are a gold standard tool for interpretable machine learning," as interpretability really depends on the context of specific problems.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

It would be beneficial to discuss the limitations of the contextual lasso, especially in relation to those of the ordinary lasso. For instance, the lasso often selects only one feature from a group of highly correlated features. Does the contextual lasso suffer a similar issue? Furthermore, how sensitive are the selected features to minor adjustment in the hyperparameter $\lambda$? Such stability considerations are important to enable better interpretation of the model.

Reviewer Eybr7/10 · confidence 3/52023-07-08

Summary

This paper proposes a contextual lasso, which makes the coefficients of features sparse depending on the context. Unlike the standard lasso, the contextual lasso regularizes prediction loss with the expectation of L1 regularization. The sparse coefficients are produced by neural networks whose last layer is a projection layer that maps dense coefficients onto l1-ball. The experimental results on synthetic and real datasets show that the contextual lasso outperforms the linear lasso and 'dense' contextual lasso while achieving the high sparseness of the coefficients.

Strengths

Overall, this paper is well-organized and well-written. Section 2, which describes the proposed method, is easy-to-follow. The proposed contextual lasso is technically sound, and the high expandability similar to the lasso is described. The experiments are organized well, and the experimental results are convincing.

Weaknesses

In the experiments, the contextual lasso is assessed only on tabular datasets with relatively low dimensional features. Therefore, it is unclear whether the contextual lasso consistently works well for high-dimensional features and data other than tabular data, such as images and texts.

Questions

- When contextual features are useful but explanatory features are uninformative, I think that the predictions using the contextual lasso would be worse than the standard neural networks, which use both contextual and explanatory features as input. Is there any solution to this? - It would be easier to read this paper if figures were displayed at the top of the page.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

I could not find the description of the limitations.

Reviewer Eybr2023-08-14

Thank you

Thank you for your response and for performing the additional experiment. The experiment has convinced me that the proposed method is likely to work well even for high-dimensional data. As I think your paper deserves to be accepted, I will keep my score.

Reviewer DGcD2023-08-14

I appreciate authors for detailed response. Response 2 For the explanatory features or contextual features, reading the rebuttal, my understanding is that when the data exhibit such a explanatory\contextual structure, the proposed method would work well. Within this realm, with the additional experiments, I think the paper makes reasonable contribution. Response 5 In my experience, I agree that best MSE or classification accuracy often attained at a dense model, so using validation set to select $\lambda$ can result in a dense model. So including results with similar sparsity level will also help demonstrate the advantage of proposed method. I am keeping my score for now and await for reviewer discussion.

Reviewer kkJW2023-08-19

Thanks.

I would like to thank the authors for answering my questions. I am keeping my original score.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC