How do Minimum-Norm Shallow Denoisers Look in Function Space?

Neural network (NN) denoisers are an essential building block in many common tasks, ranging from image reconstruction to image generation. However, the success of these models is not well understood from a theoretical perspective. In this paper, we aim to characterize the functions realized by shallow ReLU NN denoisers -- in the common theoretical setting of interpolation (i.e., zero training loss) with a minimal representation cost (i.e., minimal $\ell^2$ norm weights). First, for univariate data, we derive a closed form for the NN denoiser function, find it is contractive toward the clean data points, and prove it generalizes better than the empirical MMSE estimator at a low noise level. Next, for multivariate data, we find the NN denoiser functions in a closed form under various geometric assumptions on the training data: data contained in a low-dimensional subspace, data contained in a union of one-sided rays, or several types of simplexes. These functions decompose into a sum of simple rank-one piecewise linear interpolations aligned with edges and/or faces connecting training samples. We empirically verify this alignment phenomenon on synthetic data and real images.

Paper

References (31)

Scroll for more · 19 remaining

Similar papers

Peer review

Reviewer 7Tj76/10 · confidence 3/52023-06-28

Summary

The paper studies the shape and properties of one layer networks on a denoising problem. Specifically, the authors compute a closed-form solution for a NN trained offline with regularization and show that its ability to generalize is better than the eMMSE estimator -- which acts as a piece-wise constant function -- in a low-noise regime and in 1d. Then they provide results on multivariate cases where the training data points are contained in a lower dimensional subspace, showing that the image of the network is also contained in the subspace, and deriving closed-forms for specific cases where the data are aligned.

Strengths

Regarding the presentation, the paper is clearly written and each theoretical result is well-explained, making the reading easier. Regarding the content, the theorems seem new and provide interesting insights on the behavior of simple networks on the denoising problem. The authors provide a complete study of the univariate case when the noise level is reasonable, and give insights on what happens in higher dimensions. Numerical illustrations support the theoretical claims.

Weaknesses

The study is limited to low noise levels (noisy samples supports can't intersect), and it would have been interesting to have the authors opinion on whether removing assumption 1 change or not the behavior shown in figure 1 (theoretical development or even empirical illustrations). In particular, theorem 1 is valid in the limit when $\sigma \to 0$, while we would have preferred a "threshold" (as the authors mention, there exists a threshold for the strict inequality but it is potentially very small).

Questions

* Could the authors illustrate what happens in practice (in 1d or in higher dimension) when the noise level increases ? In particular, do we still have a gain using a NN rather than the eMMSE ? * Is it possible to give bounds on $\sigma$ for which theorem 1 is valid ? Otherwise, could the authors provide MSE curves highlighting the difference between NN and eMMSE depending on $\sigma$ (in the case where the noisy samples support are disjoint and in the general case) ?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The limitations are addressed by the authors.

Reviewer kiMs7/10 · confidence 3/52023-07-04

Summary

This paper studies how two-layer ReLU denoising networks look like when minimizing common losses such as the empirical minimum mean square error, an empirical alternative that draws finitely many noisy samples, as well as representation costs that find the data interpolating function with minimal norm of the weights. It is proven that - for univariate data - representation cost minimizers generalize better than empirical minimum mean square error minimizers in the low noise regime. Moreover, the solution to the representation costs is explicitly stated in the univariate case as well as in the multivariate case for specific geometric configurations, including one conjecture. Small numerical experiments illustrate that theory is correct and the conjecture is justified.

Strengths

- The paper is very well written and easy to follow despite its theoretical nature - It characterizes several interesting and important properties of shallow denoising networks in the low-noise regime for univariate data. - It gives inspiring insights on the behavior in the multivariate case (under particular assumptions). - It encourages further research in the investigated direction by posting one (partially open) conjecture, as well as by accurately stating several limitations (tight to very interesting directions of future research). - It contributes to our understanding of denoisers in function space

Weaknesses

I am not familiar enough with the explicit characterization of networks that minimize representation costs to judge the novelty and impact of the contribution. I was a little surprised to read "Hanin [2021] gave a full characterization of univariate representation cost minimizers subject to data interpolation constraints", and that there are follow-up extensions, which seem very relevant to the paper at hand. Beyond this, except for the limitations that have already been stated by the authors themselves, I do not see any major weaknesses. In practical terms, I am now curious to what extent larger (deeper + more sophisticated, e.g. convolutional or normalization including) models inherit some of the shown properties, but can understand that this goes beyond the scope of the paper. Also, is the M<<d problem discussed in line 333 a reason why adversarial examples exist? The strict interpolation on a ball would prevent them.

Questions

- Please clarify in what way your work extends previous characterizations of minimal representation cost solutions. - And just out of curiosity - if "Savarese et al. [2019] showed that the representation cost of a function realizable as a univariate two-layer ReLU network coincides with the L1-norm of the second derivative of the function" (and assuming that the second derivative of a ReLU network has some meaning like the total variation of the (step-function-like) first derivative), doesn't proposition 1 follow from this quite directly?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

The authors have addressed the limitations well.

Reviewer qSF75/10 · confidence 3/52023-07-06

Summary

The elementary properties of NN solutions for the denoising problem have been explored, with a focus on offline training of a one hidden-layer ReLU network. When the noisy clusters of the data samples are well-separated, there exist multiple networks with zero loss, even in the case of under-parameterization, while having a different representation cost. In the univariate case, a closed-form solution to such global minima with minimum representation cost has been derived. It has also been demonstrated that the univariate NN solution generalizes better than the eMMSE denoiser. In the multivariate case, it has been shown that the interpolating solution with minimal representation cost is aligned with the edges and/or faces connecting the clean data points in several basic cases.

Strengths

The NN solutions is studied in the setting of interpolation of noisy samples with minimal representation cost, in a practically relevant “low noise regime” where the noisy samples are well clustered. In the univariate case, a closed-form solution for the minimal representation cost NN denoiser is derived and shown to have better generalizition behavior than the empirical minimum MSE denoiser. In the multivariate case, a closed-form solution for the minimal representation cost NN denoiser in multivariate case under various assumptions on the geometric configuration of the clean training samples. Moreover, a general alignment phenomenon of minimal representation cost NN denoisers is illustrated in the multivariate setting.

Weaknesses

Weakness 1: Empirical effectiveness. This paper proposes a shallow denoiser and analyizes its theoretical performance. W1.1 Despite the interesting theoretical properties of the shallow denoiser, it is not clear whether the proposed model performs well in real image denoising applications due to the lack of sufficient empirical evaluation and analysis. W1.2 In the multi-variate settings, the authors consider training data on a subspace. However, it is not clarified in which typical settings this subspace assumption holds. Thus, the empirical reasonability of the subspace-based analysis is not well addressed. Weakness 2: The theoretical findings may be not sufficiently new. For example, the NN solution is shown contractive towards the clean data points, which has already been empirically observed in Autoencoders by Radhakrishnan et al. [2018].

Questions

Please see and address the weaknesses.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

I think the authors have well discussed the limitations.

Reviewer 4XGM6/10 · confidence 3/52023-07-06

Summary

This paper looks at the denoising problem and compares the neural network denoiser to what the paper calls the eMMSE denoiser. The eMMSE denoiser is the optimal denoiser (in function space) given finite data and known noise distribution. In contrast, the Neural Network denoiser has access to a finite number of copies of each of the finite data points and minimizes the empirical noise. The paper shows that even in simple settings (univariate data), the minimum norm neural network denoiser has better generalization than the eMMSE denoiser, as it interpolates in between the data points whereas the eMMSE denoiser acts as a nearest neighbor denoiser. The paper further explores some other settings such as low rank and data on rays.

Strengths

The main strength of the paper is finding the explicit form of the minimum norm two-layer denoiser. This is interesting as it identifies the best point in function space that can be represented as a two-layer network. Hence gives the global minimizer for the neural network which is traditionally not easy. Hence this is important. Other strengths of the paper include that they show these denoisers are contracting and generalize better than some "optimal" denoisers.

Weaknesses

The are a few weaknesses of the paper. 1) I think the presentation of the paper can be improved significantly. In many cases, the paper switches between functional representations of the function and the parametric representation of the function. While both are interesting, it would be nice to have to a clear distinction between the two and a way of translating from one representation to the other. 2) While the results of the paper are interesting, there are limitations in the types of data that they consider. Specifically, univariate data, data on a line, and data on the union of lines are the cases in which the paper can identify the min norm solutions. While these cases are interesting and I understand that more general cases are challenging. Hence I don't think it is needed to solve more general cases. However, if the paper could extract some insights from these theoretical cases that might apply to more general cases, this would help strengthen the paper. 3) I think the related works section is missing various theoretical works on denoising. [A-D] look at denoising via factorizations which fit in nicely with the low-rank structure that the paper looks at, and [E] looks at denoising regression case. [A] Raj R. Nadakuditi. OptShrink: An Algorithm for Improved Low-Rank Signal Matrix Denoising by Optimal, Data-Driven Singular Value Shrinkage. IEEE Transactions on Information Theory, 2014 [B] Marc Lelarge and Léo Miolane. Fundamental Limits of Symmetric Low-Rank Matrix Estimation. In Proceedings of the 2017 Conference on Learning Theory, 2017 [C] Antoine Maillard, Florent Krzakala, Marc Mézard, and Lenka Zdeborová. Perturbative Construction of Mean-Field Equations in Extensive-Rank Matrix Factorization and Denoising. Journal of Statistical Mechanics: Theory and Experiment, 2022 [D] Emanuele Troiani, Vittorio Erba, Florent Krzakala, Antoine Maillard, and Lenka Zdeborov’a. Optimal Denoising of Rotationally Invariant Rectangular Matrices. ArXiv, abs/2203.07752, 2022 [E] Rishi Sonthalia and Raj Rao Nadakuditi. Training data size induced double descent for denoising feedforward neural networks and the role of training noise. Transactions on Machine Learning Research, 2023

Questions

1. The paper presents the network with a skip connection, but as far as I can tell, the skip connection version is not studied in the paper. As $R(f)$ (Definition 1, Equation 19) is without the skip connection. 2. The denoiser in Corollary 1, what would be the neural network representation of that function with ReLU activation? 3. The fact that the online denoiser and the offline are equivalent is interesting. Do you have any intuition as to why the offline captures the online phenomena despite having much fewer samples?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

N/A

Reviewer 4XGM2023-08-12

Thank you for the clarifications.

I will increase my score. I did have one more question. I think why I thought (13) did not have the skip connection ($V$) is because the norm of the skip connection is not regularized. Is there a reason this is the case? Do the authors know what might happen if $V$ included in $R(f)$?

Authorsrebuttal2023-08-16

Thank you for increasing the score. We followed the same setting as in [A] and [B], which did not regularize the skip connection. Previous works also considered other cases: without skip connection ([A] Theorem 2), and regularization on the bias [C]. Adding regularization over the skip connection is an interesting case to consider, which was not directly covered in previous works: [D] and [E] proved related results but in a different setting (constrained path/optimization path in classification vs. regularization path in regression in our case). From these works, it intuitively appears that a “large” target function is “cheaper” to implement without a skip connection (recovering the case without the skip connection), since we can distribute the function scale over two layers instead of one. However, for general target functions the situation is more complicated. We can prove (see below) that the cost of realizing the linear function $L(x) = Vx$ with the regularized skip unit is greater than the cost of realizing it with regularized ReLU's if and only if $2||V||\_* \leq ||V||\_F^2$, where $||V||\_*$ is the nuclear norm. This will generally hold for large norm matrices (following the intuition above), but it fails for matrices with sufficiently small norms. We are not sure yet what holds for general (nonlinear) target functions. This is an interesting open question to explore. __Proof:__ Let $C\_{relu}$ be the minimum cost needed to realize $L(x) = Vx$ with ReLU units over some compact domain. We can write every such realization as $L(x) = A[W^Tx+b]\_+ -Ab$, where $[W^Tx+b]\_+ = W^Tx + b$ on the domain, and $V = AW^T$. This gives $C\_{relu} = min\_{V = AW^T} \sum_k (||A||\_F^2 + ||W||\_F^2)$. But by the variational characterization of the nuclear norm, we see that $C\_{relu} = 2||V||\_*$. On the other hand, to realize $L(x)$ with a regularized skip connection costs $C\_{skip} = ||V||\_F^2$. So as long as $2||V||\_* \leq ||V||\_F^2$, we have $C\_{relu} \leq C\_{skip}$. [A] Ongie, G., Willett, R., Soudry, D. and Srebro, N., 2019. A function space view of bounded norm infinite width relu nets: The multivariate case. [B] Hanin, B., 2021. Ridgeless Interpolation with Shallow ReLU Networks in $1 D $ is Nearest Neighbor Curvature Extrapolation and Provably Generalizes on Lipschitz Functions. [C] Boursier, E. and Flammarion, N., 2023. Penalising the biases in norm regularisation enforces sparsity. [D] Nacson, M.S., Gunasekar, S., Lee, J., Srebro, N. and Soudry, D., 2019, May. Lexicographic and depth-sensitive margins in homogeneous and non-homogeneous deep models. [E] Kunin, D., Yamamura, A., Ma, C. and Ganguli, S., 2022. The asymmetric maximum margin bias of quasi-homogeneous neural networks.

Reviewer qSF72023-08-14

Response to the reply

Thanks for the authors' feedback. While addressing my weakness 1.2 for the empirical reasonability of the subspace-based analysis, the authors give examples in face recognition and handwritten digits, where PCA achieves good performance. However, it is more convincing to analyze the subspace assumption on large scale datasets today like ImageNet.

Authorsrebuttal2023-08-17

As shown in [A], it is a general phenomenon that large datasets are (approximately) low rank [A], i.e., lie on a linear subsapce. Following the reviewer׳s suggestion, we also validated the subspace assumption on the following common image datasets: * CIFAR10 * CINIC10 * Tiny ImageNet (a lower resolution version of ImageNet, enabling us to use SVD) * BSD (a denoising benchmark composed from 128X1600 patches of size 40X40 cropped from 400 images [B]) We applied a Singular Value Decomposition (SVD) for each of the above datasets, and calculated the relative number of Singular Values (SV) needed to achieve a given percentile of the energy (for the average vector). __CIFAR10__: 95%, 99%, and 99.9% of the energy is concentrated on 0.8%, 7.5%, and 30% of the SV, respectively. __CINIC10__: 95%, 99%, and 99.9% of the energy is concentrated on 1%, 23%, and 41% of the SV, respectively. __Tiny ImageNet__: 95%, 99%, and 99.9% of the energy is concentrated on 1.6%, 20%, and 36% of the SV, respectively. __BSD__: 95%, 99%, and 99.9% of the energy is concentrated on 0.1%, 1.6%, and 4.5% of the SV, respectively. As can be seen from the results, the subspace assumption holds for all the datasets we used. We hope we were now able to completely address all the reviewer's concerns, if there are any remaining concerns, please let us know. [A] Udell, M. and Townsend, A., 2019. Why are big data matrices approximately low rank?. [B] Zhang, K., Zuo, W., Chen, Y., Meng, D. and Zhang, L., 2017. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising.

Reviewer qSF72023-08-21

Many thanks for the authors' feedback. I would like to keep my,score unchanged.

Reviewer 7Tj72023-08-14

Thanks for the answer

Thanks to the authors for their answer and the clarifications regarding the noise level.

Reviewer kiMs2023-08-16

Thanks

Thanks a lot for the detailed response and the additional results which further strengthen the paper! I clearly recommend the acceptance of this work!

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC