Compression with Bayesian Implicit Neural Representations

Many common types of data can be represented as functions that map coordinates to signal values, such as pixel locations to RGB values in the case of an image. Based on this view, data can be compressed by overfitting a compact neural network to its functional representation and then encoding the network weights. However, most current solutions for this are inefficient, as quantization to low-bit precision substantially degrades the reconstruction quality. To address this issue, we propose overfitting variational Bayesian neural networks to the data and compressing an approximate posterior weight sample using relative entropy coding instead of quantizing and entropy coding it. This strategy enables direct optimization of the rate-distortion performance by minimizing the $\beta$-ELBO, and target different rate-distortion trade-offs for a given network architecture by adjusting $\beta$. Moreover, we introduce an iterative algorithm for learning prior weight distributions and employ a progressive refinement process for the variational posterior that significantly enhances performance. Experiments show that our method achieves strong performance on image and audio compression while retaining simplicity.

Paper

Similar papers

Peer review

Reviewer rYFn7/10 · confidence 4/52023-06-24

Summary

## Summary The authors propose an approximated correlation communication approach to compress Bayesian INF. Two practical considerations, such as prior fitting and posterior refinement are proposed. It achieves an R-D performance comparable to the SOTA INF method on image compression. Moreover, the audio compression is also considered. ======================================== ## Post rebuttal The authors have addressed my concern wrt to the experiments, in a quite unexpected way. I raise my rating to accept.

Strengths

## Strength * The idea of using correlation communication to compress image INR is novel. The compression of bayesian INR is also novel and likely to have impact outside compression community. Furthermore, it also considers audio compression and achieves reasonable results.

Weaknesses

## Weakness * To the best of my knowledge, there is no algorithm that communicates correlation using finite number of samples. The adaptative reject sampling [Harsha 2010, The Communication Complexity of Correlation] and Possion Functional Representation [Li 2018, Strong Functional Representation Lemma and Applications to Coding Theorems] both requires infinite number of samples in the worst case to communicate exact posterior. As [Theis 2021, Algorithms for the Communication of Samples] has shown in Fig. 2, the expected code length of relative entropy coding [Flamich 2020], even if converge expotentially fast, still have quite some distance to the true posterior. And the expected code length of [Flamich 2020] is also inferior to [Harsha 2010] [Li 2018], which is $D + \log D + \log\log D +,... $ The [Flamich 2020] does not emphasis this gap, while it is supposed to negatively effect the performance of the proposed approach. And this inaccurate posterior gap also exists for A* coding [Flamich 2022]. Then my question is, how bad such posterior approximation it is? Or to say, if we assume that we can use infinite sample optimal PFR to achieve exact posterior, what is the R-D curve like? * Another weakness is that the advantage of the proposed approach over MSCN [Schwarz 2022, Meta-Learning Sparse Compression Networks] is unclear. From the perspective of R-D performance, the proposed approach is only comparable to MSCN on Kodak and is slightly outperformed by MSCN on CIFAR. However, the encoding time of the proposed approach seems to be quite long due to the progressive posterior refinement. As neither the author nor original MSCN paper report encoding time of MSCN, I can not evaluate the advantage of the proposed approach pver MSCN. Or to say, if the authors can show that the run time of propoased approach is significantly faster than MSCN, this paper is more acceptable. This is especially true as the authors claim that their approach is simpler (L54,66,253). They need evdience, such as faster runtime, to support their claim.

Questions

See weakness.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

The limtation is discussed.

Reviewer 5R2p6/10 · confidence 4/52023-06-27

Summary

This paper proposes overfitting variational Bayesian neural networks to the data and compressing an approximate posterior weight sample using relative entropy coding, which enables direct optimization of the rate-distortion performance by minimizing the $\beta$-ELBO. Moreover, an iterative algorithm for learning prior weight distributions is introduced and a progressive refinement process for the variational posterior is employed for improved performance. Experiments were conducted on image and audio compression.

Strengths

- the work encodes data with variational Bayesian implicit neural representations, which enables direct optimization of the rate-distortion performance by minimizing the $\beta$-ELBO. - the work presents an iterative algorithm for learning prior weight distributions and a progressive refinement process for the variational posterior for improved performance.

Weaknesses

The presentation can be improved, e.g., - $\theta_p$ indicates $\mu_p, \sigma_p$, which should be clearly described. - Some typos, e.g., Line 183: represent -> represents; Line 190: the to -> the; Line 209: the its -> its; Line 296: subscript "2". - Title of [5] is missed, "Auto-Encoding Variational Bayes". Similar problem in [35]. And [35] actually repeats [34].

Questions

- It would be nice to compare the proposed method with the concurrent work [52[ published in ICML 2023. - Will the encoding time be the obstacle for applying he porposed method to video compression or 3D data compression? - Code availability helps reproducibility although enough implementation details are provided.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors adequately addressed the limitations.

Reviewer 7dCG6/10 · confidence 4/52023-07-07

Summary

The paper improves INR-based (image) compression by introducing majorly two techniques: 1) a relative entropy coding based model compressing framework as an alternative to commonly used quantization - entropy coding pipeline; 2) a semi-amortized approach to train the model prior which is similar to beta-VAE and enables RD tradeoff with relative entropy coding. With the proposed novel pipeline, one can adopt COIN-like INR compression with better RD performance because it gets rid of quantization errors.

Strengths

The paper is well-written with dedicated figures and easy to follow. Overall I like the story of adopting beta-VAE/REC in INR coding methods. It is intuitive that given a dataset, all data neural representations share some common knowledge, and a potential amortized approach tends to exist. Thus, a meta-model describing the prior model distribution is soundable. This re-connects compression with VAEs, though a different way from feature-based learned image compression methods. And this re-connection makes a finer rate control of INR more promising. I believe this work contributes to society and can inspire future studies.

Weaknesses

My major concern is about practicality. Seems that to train a model prior, we should first train many model posteriors. As also discussed by the authors, this training makes the entire training (encoding) time extremely slow. Is it possible to adopt this model posterior to further speed up the encoding of samples out of the training set? i.e. we may imagine efficiently finetuning a new INR (posterior) from the obtained prior. If this finetuning requires less time to convergence, the enlarged encoding time may be amortized. Another issue for me is the somewhat marginal performance improvement. Is it worth costing such a large encoding time for the model prior, instead of simply training a COIN/COIN++ model longer to compensate for the quantization error? It would be better to report the converging speed of the models e.g. something similar to PSNR-iteration curves.

Questions

See above

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have discussed the limitations, which is convincing.

Reviewer D8f28/10 · confidence 3/52023-07-15

Summary

This paper addresses the problem of lossy data compression (evaluation is on images and audio) using implicit neural representations (INRs). In this approach, a neural network is designed that maps coordinates (e.g., x,y locations in an image or time for audio) to samples (RGB values or audio amplitude). Then the overfit network itself is the representation that is stored or transmitted since the data can be approximately recovered by running the network over appropriate coordinates. INRs for data compression are not new, but this paper includes two key novelties: (1) a variational Bayesian approach is adopted, which (2) allows the method to jointly optimize for rate (the entropy of the neural network parameters) and distortion (the quality of the reconstructed data). Previous INRs for compression used less powerful formulations and separately optimized for rate and distortion, which typically leads to worse overall performance (and the empirical evaluation in this paper demonstrates that here). Within this setup, the authors describe an iterative algorithm for learning the prior over network weights, and they describe a progressive refinement method that improves performance by dividing the network parameters into blocks and conditionally coding each block given previous blocks. The method is evaluated on images (CIFAR-10 and Kodak) and audio (LibriSpeech). In both cases, the method is shown to outperform previous INR-based models. For images, it does not outperform the best VAE-based compression methods.

Strengths

Originality: the problem is not new and jointly optimizing rate and distortion is quite common in neural compression in general, but I have not seen it done before for INRs. Nor have I seen an approach that uses variational Bayes and relative entropy coding with INRs. Quality and clarity are both excellent. I think the significance of this paper is quite high for the neural compression community and especially researchers looking at INRs. VAE-based (often called "nonlinear transform coding" or NTC) is the dominant approach right now, but INRs are quite interesting and there's a growing literature using such methods for compression. This work is significant because it does the right thing (joint optimization of rate and distortion), which has not been done before with INRs, and because it sets a new SOTA for compression with this approach.

Weaknesses

1. Obviously the paper would be stronger if the empirical results were better. Focusing on Fig. 2b (RD curves on the Kodak image set), COMBINER trails CVPR2020 by more than 2dB at 0.2 bpp (a huge gap), and CVPR2020 is no longer a SOTA approach. That said, COMBINER is SOTA for INR-based methods (as far as I know), which is an important result, and I think research can get stuck in a local minimum if we reviewers require new approaches to outperform established ones too soon in the research cycle, e.g., Fig. 2b shows that COMBINER slightly outperforms the VAE-based method from ICLR2018. 2. Neural model compression is a large subfield so some discussion or comparison with a model compression method (beyond simple weight quantization) seems like an important baseline that's missing.

Questions

Section 5.1 ("Models") discusses the model architecture saying that it was empirically determined. How sensitive are the results to the base model architecture, and how easily can the encoding process prune (reduce parameter entropy to near zero) large parts of a model architecture that is much larger than the RD optimal model?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

adequately addressed

Reviewer WLVL5/10 · confidence 2/52023-07-27

Summary

The paper proposes a new method for compressing general signals, by using Variational Bayesian implicit neural representations. It proposes an algorithm for learning a prior distribution over the implicit representation weights, as well as a pipeline for inferring the posterior distribution corresponding to every given signal to be encoded. The authors employ several tricks (e.g. Relative Entropy Coding) borrowed from previous works, that are meant to increase compression efficiency. The method is evaluated over two image datasets (namely the CIFAR10 dataset of very small images and the Kodak dataset containing 24 larger images) and one audio dataset, while comparing performance with several image compression methods and the classic MP3 audio compression method, respectively.

Strengths

Generally speaking, the paper is presented well - despite the fact that compression is not my main expertise, I was able to follow and understand most of it. The combination of the different ideas used in this method is interesting and appealing. Limitations, as well as concurrent work, are discussed candidly.

Weaknesses

Besides the method's complexity, which could (at least partially) be attributed to my lack of expertise in this field, I found two major flows: Performance: The performance curves presented in Fig. 2 and 5 in the paper do not indicate an advantage of the proposed method over many of the existing methods, in terms of the rate-distortion trade-off. In the case of audio (Fig. 5), some advantage is shown only over the classic MP3 compression, and only for part of the kbps range. Speed/Complexity: The encoding process of this method is long and expensive. Besides being a big limitation (as noted by the authors) and hence a weakness of the method compared to some of the other ones, I'm missing some detailed comparison of these aspects. A third, relatively more minor flow has to do with the experimental setup itself, as experiments were conducted over only two small image datasets and one audio dataset, and compared only with MP3 in the latter case.

Questions

As I'm not entirely familiar with this field, the authors are welcome to direct my attention to certain points they feel that I overlooked, and I'll gladly consider increasing my initial rating.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Limitations were candidly discussed in the paper.

Reviewer 5R2p2023-08-14

Thank you for the response

Thank you for the response.

Reviewer rYFn2023-08-14

* First of all, if exact posterior is used, the the rate shuold be $D_{KL} + \log D_{KL} + \log\log D_{KL}...$ bits. If the current curve of Figure 2 in the rebuttal document, it is wrong, as its rate is $D_{KL}$ bits, which is only true for asymptomatic channel simulation, not one shot channel simulation like this paper. I suggest the author to consider this one-shot overhead and show the correct figure in the later version. * Second, I am glad to see that the authors address my concern related to MSCN, in a very very unexpected way. Currently I am more confident with accepting this paper, and I will raise my rating to accept.

Authorsrebuttal2023-08-14

Response to Reviewer rYFn

We thank the reviewer for their response to our rebuttal; we address the reviewers first comment below > First of all, if exact posterior is used, the the rate shuold be $D_{KL} + \log D_{KL} + \log\log D_{KL} + \dots$ bits. If the current curve of Figure 2 in the rebuttal document, it is wrong, as its rate is $D_{KL}$ bits, which is only true for asymptomatic channel simulation, not one shot channel simulation like this paper. I suggest the author to consider this one-shot overhead and show the correct figure in the later version. We believe there are two points of slight misunderstanding here, which we address below. Please let us know if we did not interpret the reviewer's comment correctly. **The reported curve in Figure 2:** We believe the reviewer assumes that the dashed line in Figure 2 is supposed to describe the efficiency of an exact channel simulation protocol, for which the codelength would include some additional overhead terms besides the KL divergence, as the reviewer rightly mentions. However, this is not our intention. As described in Section 3.3 of the paper, we use an approximate protocol such that the codelength is always approximately equal to the KL (within each block). Hence, what we compare in Figure 2 is the quality of approximate samples compared to exact ones at the same coding cost. In other words, the dashed line describes the practically unattainable scenario if our scheme would always yield exact samples instead of approximate ones at the same codelength as before. We also note that if we included the extra log term in the rate calculation of the idealised sampler, the theoretically ideal curve would look worse since it would shift the curve to the right; thereby making COMBINER look better. **The overhead terms in exact channel simulation:** We believe the reviewer is referring to the limiting case of Li and Vitanyi's universal prefix-free codes to encode the index returned by the sampler, yielding a codelength bound of approximately $D_{KL} + \log D_{KL} + \log\log D_{KL} + \dots$ in the one-shot case. Via Jensen's inequality, this would yield a bound of approximately $I[\mathcal{D} : w] + \log I[\mathcal{D} : w] + \log\log I[\mathcal{D} : w] + \dots$ in the average case, where $I[\mathcal{D} : w]$ denotes the mutual information between the data $\mathcal{D}$ and the INR weights $w$ that encode it. However, using the zeta distribution approach described in Appendices A and B in Li & El Gamal (2018), we can reduce the average-case bound to approximately $I[\mathcal{D} : w] + \log (I[\mathcal{D} : w] + 1) + 5$, which is a better bound! ## References M. Li and P. Vitanyi, An Introduction to Kolmogorov Complexity and Its Applications, 3rd ed. New York: Springer-Verlag, 2008. C. T. Li and A. El Gamal (2018). Strong functional representation lemma and applications to coding theorems. IEEE Transactions on Information Theory, 64(11), 6967-6978.

Reviewer WLVL2023-08-17

Thank you for your response. I now feel comfortable raising my score, following your clarification regarding the distinction with regard to INR-based methods (due to their relative novelty). However, as this argument still does not explain the slight disadvantage with respect to the INR-based MSCN method on the CIFAR dataset, and since it's unclear whether the proposed method is better than MSCN in terms of speed (as noted by reviewer rYFn), I'll settle for borderline accept.

Reviewer 7dCG2023-08-21

The authors have addressed my concerns. I maintain my acceptance rating.

Reviewer D8f22023-08-22

> Havasi et al. [23] applied relative entropy coding to model compression and demonstrated state-of-the-art performance... The authors' reliance on previous results comparing relative entropy coding to other model compression methods seems sufficient. It would be good to add a reference to (Oktay 2020) and make it clear in the paper that they're relying on results from that paper and (Havasi 2019). I don't see any reason to adjust my rating.

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC