Bayesian Online Natural Gradient (BONG)

Summary

The paper introduces a framework for variational online learning, in which a method is defined by three ingredients: (1) implicit or explicit regularisation (2) type of approximation of the expected gradient and hessian (3) geometry in which the gradient is taken (ie employing natural gradients or standard gradients). In implicit regularisation, one learning step is taken for each datapoint, with no regularisation; explicit method is the more standard approach of optimising with multiple steps and regularisation to the posterior at the previous step. A case study on different combinations of the ingredients is conducted; some of the methods appeared in previous papers. The study finds the best performing method is the one with implicit regularisation, linearisation of the hessian, and natural gradient.

Strengths

The paper is a valuable contribution to the field, offering a framework for disjointed methods. It reads like a review paper in variational online learning, with a novel contribution based on the review. The list of methods of approximation is thorough and the comparison exhaustive.

Weaknesses

Given that the main strength of the paper is the unifying recipe, and the comparison of using different ingredients, a more exhaustive case study would strengthen the work (and the less essential comments, like the equivalence to the full Bayesian update in the conjugate case, in my opinion can be placed in the appendix and referenced). Nevertheless, I find the contribution of the paper warrants it being accepted.

Questions

* Given the well documented efficiency of natural gradient steps, is there a danger of the method with no regularisation being more prone to "forgetting"?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

N/A

Summary

The authors explore sequential Bayesian inference using variational inference (VI). They propose to remove the regularizing KL term, performing a single natural gradient step using the expected log-likelihood only, and provide relevant ways to approximate the hard-to-compute expectations. They support their approach with detailed experiments.

Strengths

The paper is sound, and the detailed experiments provide strong support for the method proposed in the paper. I also appreciated the nice motivation involving conjugacy, as well as the algorithmic considerations.

Weaknesses

My main concern is about the claimed novelty of the approach ("removing the KL" and "implicit regularization towards the prior"). The idea of using a one-step natural gradient descent on the expected log-loss directly (without the KL term) has already been proposed in the literature and analyzed; for example, [Lyu & Tsang, 2021] studied this algorithm in a model-free setting where a general function f is considered instead of the log-loss. This one-step natural gradient update is in fact simply the solution of Equation (2) of the present paper, but with a linearized expected log-loss. This dual interpretation resonates with the dual interpretation of standard Euclidean gradient descent (single update or minimization of a first-order development of the loss to be minimized). Y. Lyu and I. W. Tsang: Black-box optimizer with stochastic implicit natural gradient. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 217–232, 2021.

Questions

NA

Rating

6

Confidence

3

Soundness

4

Presentation

3

Contribution

3

Limitations

NA

Summary

The paper proposed a novel parameter update rule. The rule is natural gradient descent on the variational inference loss, where the prior term is dropped. Such rule is motivated in theory under the assumption of likelihood and variational distribution being conjugate, and in practice though experiments on MNIST. Several variants are considered: various loss, various hessian approximations and various way of dealing with the intractable expectation.

Strengths

The paper is well written and clear. The setting is explained properly and the related work are exhaustive. A lot of variants of the algorithm are considered. These are well explained and the previously known methods are highlighted. This schematic approach is very valuable for two reasons: \ -it helps in better understanding the proposed algorithms \ -it creates a clear picture of the field of approximated bayesian inference, putting in perspective every method with each other In my opinion this is a contribution that may be worth acceptance, not the proposed update rule.

Weaknesses

The theoretical motivation is quite weak:\ -Proposition 4.1 is nice, but the assumptions are overly restrictive. It feels like this is such a special case that is has no implication on real case scenarios \ -The claimed contribution (in Line 44) is a very weak contribution. The closed from expression with linearization+gaussian likelihood was already observed by Immer, and I don't think it is fair to claim it as a novel contribution of the paper. Specifically because there is no motivation to such approximations. On the other hand, as the authors also correctly point out in Line 307, the empirical evaluation is based on MNIST only. A very small dataset which doesn't really motivate the superiority of the proposed algorithm. Overall, I don't think the authors provided enough evidence (either theoretical or empirical) of the good performance of the proposed update rule. TYPOS: \ -Line 126. I think the $\theta_{-1}$ was supposed to be a $\theta{t-1}$ \ -There is a duplicate reference: Line 336 and Line 340 are the same

Questions

The notation $f_t$ that appears in Equations (9) and (10) was never introduced. I guess it refers to the evaluation of $f$ on $x_t$, is it correct?

Rating

4

Confidence

3

Soundness

2

Presentation

3

Contribution

1

Limitations

Limitations are well discussed. Good job on this point

Summary

This paper proposes a new approximate Bayesian technique specifically for online learning whereby posterior distributions over modeling parameters at time $t$ can be achieved with a single natural gradient step evaluated on a model using the previous posterior distribution at time $t-1$ as the prior distribution. This method is rigorously derived in a conjugate setting with an exponential family and extended approximately with various ablations to more expressive black-box settings. Extensive empirical studies are conducted validating the performance of the method.

Strengths

An interesting connection is made for online learning in the conjugate setting allowing for dropping the KL regularization term and producing posterior updates with a single natural gradient step. As mentioned in the summary, various ablations are explored which trade-off precision and computation speed. Finally, many extensive experiments are conducted comparing this method to other Bayesian online learning approaches.

Weaknesses

The main weakness of the method is the lack of experiments evaluating the resulting model uncertainty achieved after training. From what I read, it seems to only be predictive performance via misclassification and negative log predictive density. Granted, the latter does have some ablations where it is integrated over posterior samples; however, the presentation of these results are a bit difficult to interpret other than just methods with lower values being better. It would be interesting to investigate the predictive uncertainty via expected calibration error, or some other calibration metric.

Questions

No other questions, please address the main weaknesses. Please let me know if I missed or misunderstood anything; I am more than happy to be convinced otherwise.

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

4

Limitations

The authors adequately addressed the limitations of the proposed method.

Summary

This paper proposes a generalized framework for variational sequential inference of Bayesian neural networks with Gaussian prior distributions by further approximating the KL-divergence and expectations. It presents a very elegant unification of a wide range of (approximate) variational inference algorithms for Bayesian neural networks developed over the past 10 years and contains an extensive empirical evaluation on subsets of the MNIST dataset.

Strengths

The strongest point of the paper is the unification of many previously presented approximations as well as the methodological evaluation and comparison among them.

Weaknesses

The space of 9 pages is way to short to do this full justice and I often had to read into the 30 pages (!) appendix. I recommend the authors to further considering a journal submission.

Questions

Page 4: The abbreviation PSD as "positive-semi-definite" has never been introduced. Page 5: jac has not been introduced Page 7: In line 239, the component-wise product of \sigma^{-2} and \mu should be indicated somewhere

Rating

7

Confidence

4

Soundness

4

Presentation

3

Contribution

4

Limitations

This is not applicable.