Sourcerer: Sample-based Maximum Entropy Source Distribution Estimation

Summary

This paper introduces a new algorithm for source estimation, the task of estimating a model's parameters probability distributions consistent with observations. In opposition to previous algorithms, the proposed method is sampling-based and can hence be used when the model is a simulator, implicitly defining the likelihood even if the observations are high-dimensional. The authors highlight that several probability distributions might be consistent with the observations and advocate favoring the one with the highest entropy. Their method provides a way to target high-entropy solutions and shows that this new objective leads to a unique optimum.

Strengths

### Originality * The method is new to me. ### Quality * I did not identify flaws in the mathematical derivations. * The method is evaluated on a wide variety of benchmarks, and several runs have been made. The experiments showcase that the entropy regularization indeed leads to higher entropy sources. ### Clarity * All the necessary details are provided * The paper is well articulated. * The method is clearly explained * It is clearly stated how the method fits in the literature. * Figures are clean. ### Significance * Source estimation is a task that has received attention in recent years. * This paper fills a gap in the literature by providing a method that works with high-dimensional simulators.

Weaknesses

### Originality * I have no concerns regarding originality. ### Quality * I find that the Lotka-Volterra and SIR experiments are not totally convincing. First, it lacks comparison with NEB, which is claimed to perform poorly in such settings. I agree that NEB will probably provide poor results, but this should still be shown empirically, in my opinion. Second, the Wasserstein distance is hard to interpret. It is claimed that a low value is observed, but that is not clear to me why this value can be considered a low value as there are no baselines to compare with. ### Clarity * I have no concerns regarding clarity. ### Significance * I have no concerns regarding significance.

Questions

* Is there something that prevents you from comparing against NEB in Figure 4?

Rating

7

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

The authors have adequately addressed the limitations.

Summary

This paper deals with the problem of identifying the distribution of a source variable $s$ that generates observations $x$. ## The problem The authors propose to minimize a classical objective consisting of two terms: 1. A reconstruction term, that encourages the recovered source distribution to induce a distribution over observations that is close to the data distribution, in a certain divergence (here, the sliced Wasserstein) 2. A regularization term, that encourages the recovered source distribution to be close to a reference distribution, in a certain divergence (here, the reverse KL). Many different source distributions can minimize the first term alone, which motivates the second term to contrain the solution space. ## The probability model The probability model $p(x)$ can be expressed in terms of two other model distributions: the source distribution $p(s)$ and the likelihood $p(x | s)$. Term (1) uses a (sliced) Wasserstein divergence. Computing this divergence requires data samples and samples from the model in two steps: $s \sim p(s)$ then $x | s \sim p(x | s)$. Differentiating through this divergence requires the probability model, and therefore $p(s)$ and $p(x | s)$, to be samplable and the sampling process should be differentiable. Term (2) uses a reverse KL divergence. Computing this divergence requires evaluating the densities of the model and reference distributions, as well as samples from the model in two steps: $s \sim p(s)$ then $x | s \sim p(x | s)$. Differentiating through this divergence requires the probability model, and therefore $p(s)$ and $p(x | s)$, to be samplable and the sampling process should be differentiable. Additionally, the model density should be known or else a differentiable approximation of the divergence (the authors use the Kozachenko-Leonenko estimator) should be available. ## Numerics The authors validate their method using synthetic data where the source is known, and real data where the observations are neurophyisiological recordings and the likelihood is given by a mechanistic model.

Strengths

The presentation is clear and well-explained.

Weaknesses

Overall, I have two concerns. 1. The computational constraints on the model are heavy The choices of divergences (Sliced-Wasserstein and reverse KL) in the cost function impose a number of computational constraints of the model. Term 1 in the cost uses a vanilla Sliced-Wasserstein divergence estimator that is sample-based. This means the likelihood model $p(x | s)$ should be samplable and the sampling process should be differentiable. The authors propose to train a differentiable approximator of a black-box simulator. To be fair, the authors acknowledge that this is a general issue beyond their paper. Term 2 in the cost uses a vanilla KL divergence estimator which requires knowing the density of the model, but the authors circumvent this by using another estimator named "Kozachenko-Leonenko". Given that the authors already used a sample-based divergence in Term 1, wouldn't using another sample-based divergence in the Term 2 (e.g. the Sliced-Wasserstein again, or an MMD) between the model and reference distributions not add any further computational constraints on the model? 2. The evaluation procedure or goal remains unclear to me The framework of minimizing a cost made up of a reconstruction and regularization term is well-studied in optimization. Once could imagine two natural evaluation procedures: - how well the true sources are recovered? But the authors are clear that designing an identifiable probability model where the sources can be recovered (up to acceptable indeterminacies) is not the goal here. - how efficient is their method on a new test dataset, as compared to other similar methods? It seems to be that the authors answer the latter, but it is not clear to me what are the other methods to benchmark against.

Questions

Could the authors address the weaknesses part?

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes.

Summary

This paper proposes an approach to estimating a maximum-entropy source distribution (akin to a prior distribution over simulator parameters) for a given set of observations and simulation model. Their method assumes a differentiable simulator that may be deterministic or stochastic, and it uses neural samplers to approximate the prior and a variational objective that encourages proximity of both the marginal likelihood to the true data distribution and the estimated source distribution to some known prior distribution (which may be uniform, corresponding to entropy regularisation). In particular, proximity to the true data distribution is measured with a sliced Wasserstein distance, due to its fast computation and differentiability (preserving the differentiable pipeline). The authors present experiments on four benchmark tasks, before extending to higher dimensional and more complex examples such as the single-compartment Hodgkin-Huxley model.

Strengths

*Originality* The authors consider using a sample based loss to capture mismatch between the simulated and true data distributions, in contrast to using likelihood-based notions of distance. This is useful since simulation models often lack tractable likelihood functions. *Quality* The experimental section presents good and extensive empirical testing on a number of toy benchmark models, in addition to two additional more complex simulation models. Their approach is also well-motivated via Proposition 2.1. *Clarity* The clarity of writing is generally good. There are a few minor errors in the writing that made it not absolutely clear all the way through: - The last sentence of the first paragraph of Section 1 isn't a full sentence (or, if it is, then it not well-written because I have read it multiple times and could not turn it into a full sentence in my head). - There is a subscript $\phi$ missing in Equation 4. *Significance* I think this paper will be of some interest to the community and the techniques presented used by practitioners.

Weaknesses

My main concern is that the work presented looks like it is not a large or significant contribution beyond what is already present in the literature. From my perspective, it looks like someone could reasonably argue that the contribution is just to use a different sample-based loss function to capture the discrepancy between the data distribution and simulator's marginal likelihood. It also isn't quite clear to me how exactly the approach that uses the surrogate model (Sur. in Table 1) is different from what is presented in Vandegar et al. (2020).

Questions

Could the authors provide some more detail and clarification on how their approach differs/relates to/builds on previous works please? Ideally at least those papers already cited such as Vandegar et al. (2020), but I think some additional related literature is also missing, such as references [1] (relevant since it discusses variational approaches to targeting generalised Bayesian posteriors) and [2] (relevant to the problem of source estimation in complex simulation models) below. Is there anything that we lose by using something like the Sliced Wasserstein Distance instead of a likelihood-based discrepancy? [1] _Knoblauch, J., Jewson, J., & Damoulas, T. (2022). An optimization-centric view on Bayes' rule: Reviewing and generalizing variational inference. Journal of Machine Learning Research, 23(132), 1-109._ [2] _Dyer, J., Quera-Bofarull, A., Bishop, N., Farmer, J. D., Calinescu, A., & Wooldridge, M. (2024, May). Population Synthesis as Scenario Generation for Simulation-based Planning under Uncertainty. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (pp. 490-498)._

Rating

5

Confidence

2

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors provide a good discussion of the limitations towards the end of the paper. Perhaps something additional to discuss would be limitations associated with simulators that involve discrete randomness, which (as far as I can tell) aren't tested in this paper but which often appears in simulation modelling and presents an additional complication when the requirement is that sampling from the simulator is a differentiable operation (see e.g. [1] below). I also do not see any problems related to the broader social impact of this work. [1] _Arya, G., Schauer, M., Schäfer, F., & Rackauckas, C. (2022). Automatic differentiation of programs with discrete randomness. Advances in Neural Information Processing Systems, 35, 10435-10447._

Summary

The authors propose to use the maximum entropy principle (possibly tempered with a prior) in order to reduce the ambiguity in solving the problem of source distribution estimation. They propose to use a sample-based technique that optimizes for the Sliced-Wasserstein distance measuring the discrepancy between the simulated data and the real data. They propose experiments on both synthetic and real-world models.

Strengths

- The paper is extremely clear, concepts are presented consequentially and the method and experiments are easy to follow - The method is simple, but principled and effective on both synthetic and realistic data - The paper represents an interesting contribution to the field of source estimation methods.

Weaknesses

I do not see any major weakness in the work. A minor one: - W1: it would be interesting to see a more thorough study for models with even higher dimensionality, and how the proposed method's performance scales.

Questions

- Q1: In some lines authors frame as the maximum entropy principle as something intrinsically beneficial. As they already discuss, this can be modulated by using a reference distribution. Maybe some statements that make the maximum entropy principle stand out as a sort of gold standard could be toned down a bit? - Q2: Would it be possible for the authors to show what could happen with the choice of other or wrong distance types? - Q3: An interesting case the authors could discuss is when multiple losses may be required for different subsets of parameters?

Rating

7

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

The authors already address the limitations, or the limitations are intrinsic to the field and cannot be attributed to the proposed methodology.

Summary

The paper introduces “Sourcerer,” a new method for source distribution estimation, focusing on maximum entropy distributions to effectively handle ill-posed problems common in simulating scientific phenomena. This approach leverages the Sliced-Wasserstein distance for sample-based evaluation, offering a significant advantage for simulators with intractable likelihoods, and is demonstrated to recover high-entropy source distributions without sacrificing simulation fidelity across various tasks.

Strengths

- Utilizing the Sliced-Wasserstein distance for evaluating distribution discrepancies enables the method to operate effectively with simulators that have intractable likelihoods. - The approach is rigorously tested across multiple scenarios, including high-dimensional observation spaces and complex simulators. The results demonstrate that it not only maintains high fidelity in reproducing the empirical distributions but also achieves higher entropy in estimated source distributions compared to existing methods.

Weaknesses

- The proposed method only applies to differential simulators. I know a lot of these problems in computational biology have non differentiable simulators, as well as the Ising models. This is the drawback of the proposed method. - I also wish there is more theoretical results in terms of the statistical consistency and error bounds of using the sliced Wasserstein distance. Because the sliced Wasserstein distance is approximated via empirical measures. The estimation error is highly depending on the sample size. - I'm not sure if the maximum entropy approach is aligned with the intuition. What if the source distribution is highly concentrated in an area with high prob mass? I know $\lambda$ can be tuned to control it but will it affect the original problem a lot?

Questions

- what is the motivation of using the sliced Wasserstein distance? There are other types of distances can be used for this problem. For example, the maximum-mean discrepancy (MMD) and the kernelized Stein discrepancy (KSD). MMD and KSD also have their sliced versions. - Is there any principle of how to choose the hyperparameter $\lambda$? - I know the sliced Wasserstein distance is not really informative because it is estimated via samples thus the density information is a sort of missing. Is there any empirical result of the convergence speeds for the proposed method vs the benchmark NEB?

Rating

4

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

see the above.

Response to rebuttal

I have read both the rebuttal and the other reviewers' concern. I do not think the lack of differentiability of the simulators is a concern, as indicated by the authors in the rebuttal. I also appreciate the comparison with NEB and additional experiments on convergence. Given the rebuttal addresses satisfactorily all the points I raised and I do not find the points raised by other reviewers concerning I maintain my positive assessment of the work.

Answer to authors

I thank the authors for their clarifications and am happy to raise my score accordingly.