Differentiable and Stable Long-Range Tracking of Multiple Posterior Modes

Summary

The paper proposes the "importance weighted samples gradient (IWSG)" estimator and describes its integration into a "mixture density particle filter (MDPF)" for state space architectures. Similar to regularized particle filters, the MDPF framework represents the continuous state posterior with a continuous mixture density, which is differentiable and can be trained end-to-end using the proposed IWSG. As a discriminative approach, the proposed particle filter does not require the specification of a generative model and, in contrast to other discriminative particle filters, returns unbiased and low-variance gradient estimates.

Strengths

- Efficient and accurate particle-based estimation of complex posteriors in sequence models is an important topic and has a long history in the machine learning community. - The paper includes an easy-to-follow recap of particle filtering and puts the proposed approach in context with many relevant discriminative particle filters, including a thorough discussion of their weaknesses that motivates both IWSG and MDPF. - The proposed discriminative particle filter tackles a number of well-known challenges, such as differentiation through the resampling step, biased or high-variance gradient estimates, and limited expressiveness of the posterior parameterization. - The experiments compare the proposed particle filter to a broad spectrum of relevant baselines and demonstrate MDPFs superior performance in three state estimation tasks.

Weaknesses

- One of my concerns with this paper is its confusing presentation. The text often jumps between motivation, technical background, and related work, making it difficult to follow the intended train of thought. This problem is further aggravated by the fact that there are multiple technical streams: generative vs. discriminative particle filters, unbiased/low-variance gradient estimation, and inference in mixture models. There are many balls in the air and the paper has a hard time telling a streamlined story. Unfortunately, these are not the only problems with the presentation: Section 4 mixes technical background, the paper’s first main contribution (IWSG), and an experiment; Section 5 mixes the paper’s second main contribution (MDPF) with related work and contains no mathematical description of the model. - The figures are weak for multiple reasons: (1) the font size is much too small; (2) the main text and the figure captions cannot be read independently, because the figures contain critical information that is not explained anywhere in the text; (3) the lack of subfigures ((a),(b),…) makes it difficult to map the information in the captions to individual subplots; and (4) the figures are not placed on the page they are referenced. Figure 4 is especially problematic as it describes the core of the paper’s contribution but is impossible to understand without further context (which the text does not provide either). - The paper’s weak presentation also makes it difficult to assess the two contributions (IWSG and MDPF): (1) l.170 mentions that the proposed IWSG estimator is a continuous variant of the discrete estimator used in [6], but the exact relationship between the two remains unclear. Are there non-trivial challenges that prevent a direct generalization of [6] to a continuous setting?; (2) l.89/l.209 mentions that the proposed MDPF is a variant of [14,15], but again the differences are not explained in enough detail. Are there non-trivial challenges that prevent a direct application of [14,15] in a discriminative setting? - One of the paper’s main claims is MDPF’s ability to compute unbiased and low-variance gradients. Unfortunately, neither the technical sections nor the experiments provide any *direct* evidence supporting this claim. I would have liked to see either a technical analysis of the proposed estimator's properties or experiments that go beyond the evaluation of NLL/RMSE and at least give empirical insights along those lines. Overall, I feel this paper would benefit from another round of polishing before publication, including a reworked presentation and better positioning and differentiation of its contributions.

Questions

- Section 4: It is mentioned that the optimal transport methods described in [44,45] are compatible with Gaussian mixtures (l.154f), so I’m curious why they are not part of the evaluated methods? - Section 6: The design of the architecture used in the experiments (Sections 6.1-6.3) is not clear enough. Are all experiments based on Figure 4? If so, what are the encoders and transformations? - Section 6: One of the arguments against [5] is its use of truncated gradients (l.118f), however, l.244 mentions that the experiments use truncated BPTT. Doesn’t that bias the MDPF gradients as well and makes the arguments in favour of MDPF obsolete? - Section 6: I’m curious how the number of particles affects the reported performance metrics, and especially if the observed trends remain the same if the number of particles is significantly increased or decreased. If the authors have additional results along those lines it would be interesting to see them.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

- I commend the authors for including a paragraph on the limitations of the proposed particle filter, such as its supervised nature and computational complexity on high-dimensional data. - The paper does not discuss any ethical concerns related to the proposed method.

Summary

This paper discusses the limitations of traditional particle filters in representing multiple posterior modes and their applicability to high-dimensional observations like images. Instead, the authors propose a method that leverages training data to learn particle-based representations of uncertainty using deep neural network encoders. The authors address the issue of biased learning and heuristic relaxations by representing posteriors as continuous mixture densities, allowing for unbiased and low-variance gradient estimates.

Strengths

- This paper discusses an interesting problem. - It is easy to read and follow the train of thought. - Literature review is sufficient to grasp the idea of the idea of the paper.

Weaknesses

- As MDPF relies on DNN parameterizations for both dynamic and measurement models, this introduces significant complexity to the model which can make training and inference more computationally expensive. - Additional complexity of the model due to decoupling for A-MDPF as it requires careful tuning for separate bandwidth parameters. - While the paper mentions that bandwidth parameters can be learned through end-to-end training, optimizing this parameter efficiently can require careful tuning and thus is challenging. - I am still a bit uncertain about the authors' claim regarding the use of smaller NNs and non-learned operations in constructing the dynamic and measurement models. - Lack of comparison with alternative frameworks for tracking and robot localizations and mostly applying to synthetic data. - The limitations of the simulation setup in evaluating the MDPF and A-MDPF algorithms can be identified as follows: * The state estimation tasks focus on a 3D state consisting of translation and angle components (s = (x, y, θ)). While this simplification allows for manageable simulations, it may not capture the full complexity of state estimation in more challenging real-world scenarios with higher-dimensional state spaces. * The evaluation compares the MDPF and A-MDPF algorithms with a few selected particle filter variants (TG-PF, SR-PF, OT-PF, DIS-PF, C-PF) and an LSTM model. While this provides some insight into the performance of the proposed methods, a more comprehensive comparison with a wider range of state estimation algorithms would provide a better understanding of their strengths and weaknesses. * The training data uses sparsely labeled true states every 4th time-step, which may not accurately represent the labeling constraints in real-world applications. Dense labeling is crucial for accurately assessing the performance of state estimation algorithms. Although the evaluation uses densely labeled datasets, the training process may not fully exploit the benefits of dense labeling.

Questions

Questions regarding the proposed importance weighted samples gradient (IWSG) estimator: 1- How effective is the IWSG estimator in reducing variance compared to other gradient estimation techniques? 2- Are there scenarios or specific mixture distributions where the IWSG estimator may still suffer from high variance? 3- Have there been any empirical evaluations to quantify the variance reduction achieved by the IWSG estimator? 4- How does the choice of the proposal distribution, q(z), affect the accuracy and efficiency of the IWSG estimator? 5- What is the computational cost associated with using the IWSG estimator compared to other gradient estimation methods? Questions regarding the Algorithms: 1- How does the complexity of the deep neural networks used in MDPF and A-MDPF impact the computational requirements for training and inference? 2- Are there any strategies or techniques to improve the computational efficiency of these models? 3- How does the performance of MDPF and A-MDPF depend on the quality, size, and representativeness of the training data? 4- What happens when there is limited or biased training data available? 5- Can the learned parameters and decisions made by MDPF and A-MDPF be interpreted and understood? 6- Are there any techniques or approaches to enhance the interpretability of these models? 7- How does the performance of MDPF and A-MDPF depend on the choice and optimization of the bandwidth parameter (β) for kernel density estimation? 8- Are there any alternative methods or strategies to optimize this parameter effectively? 9- How does the decoupling of mixtures used for particle resampling and posterior state estimation affect the performance and training of A-MDPF? 10- Are there specific scenarios or domains where A-MDPF offers significant advantages over MDPF? 11- How well do MDPF and A-MDPF generalize to different problem domains or scenarios? 12- Have these models been tested on real-world datasets, and how do they perform compared to alternative approaches in those contexts?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Please refer to the weakness and question.

Summary

The paper studies the problem of sequential state estimation using discriminative particle filters. Particle filters (or SMC) methods are widely used in this setting owing to their flexibility. Recently, inspired by the success of deep learning enabled by end-to-end differentiable methods, there has been interest in imparting similar properties to particle filters for improving performance, in particular when modelling temporally extended systems. The authors first review some existing approaches for particle filtering with a focus on regularized particle filters as well approaches for making the non-differentiable resampling step differentiable. The authors highlight several drawbacks with the existing methods with differentiable resampling, including biased gradients, numerical instability and high variance gradient estimates. The proposed method is based on leveraging implicit reparameterization gradients coupled with an importance sampling scheme to obtain an unbiased gradient estimator for continuous mixture distributions. The implicit reparameterization avoids the inversion of the standardization function and the importance sampling scheme reduces the variance of the estimates. The approach avoid various pitfalls of previous attempts to make the resampling step differentiable. The authors leverage this estimator in their Mixture Density Particle Filter method, which is evaluated through a wide variety of experiments with thorough quantitative and qualitative analysis - achieving improved performance across tasks.

Strengths

* Sequential Monte Carlo approaches are widely used across various domains. Improvements to the algorithm can have a broad impact. The importance-weighted gradient estimator proposed in the paper can improve the performance on a wide-variety of tasks, beyond those studied in the paper. * The proposed approach is principled and novel, leveraging recent advances in implicit reparameterization along with classical insights from the sampling literature to improve the performance of regularized particle-filters. * The resulting method MDPF is conceptually simple, and can be incorporated easily as an alternative to existing regularized PF approaches. * The paper is quite well written with a nice review of existing approaches and a lot of useful details about the experiments. I do think the introduction can be improved a bit.

Weaknesses

* A key weakness of the approach is scaling to higher dimensional problems. It is a well known problem that importance sampling based estimators can still have high variance induced by some bad samples with high weights. (This is already discussed in the paper) * While I appreciate the thorough experiments in the paper one issue is that most of the experiments are on relatively low dimensional problems. * Although the details of the experiments are covered fairly well, the absence of code could be a problem for reproducibility efforts.

Questions

* What would be potential solutions to scaling the method for higher dimensional problems? * Have you tried the approach on some standard SMC tasks [1]? Does the gradient estimator improve performance there? [1] An introduction to Sequential Monte Carlo by Nicolas Chopin and Omiros Papaspiliopoulos. https://particles-sequential-monte-carlo-in-python.readthedocs.io/en/latest/

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The major limitations of the method are already highlighted in the paper - namely scaling to higher dimensional problems and reliance on labelled data.

Summary

The paper consider the problem of particle filter (PF)-based state estimation in nonlinear models with unknown dynamics and (discriminative) observation models. The key challenge they address is the typical inability of traditional gradient learning approaches, applied to PF, to deal with (backprop through) the non-differentiable particle resampling stage. To do this, the authors propose a novel Importance Weighted Samples Gradient (IWSG) Estimator, based on a kernel-density representation of the resampling step. The IWSG is fully differentiable but is also unbiased and, in practice, low variance, particularly compared to the previously proposed Implicit Reparameterization Gradients (IRG) estimator. The authors conduct an extensive set of experiments on simulated data (synthetic + simulator object-in-a-maze tracking scenarios) to demonstrate the utility of their IWSG. IWSG is shown to outperform the competing approaches.

Strengths

+ Well-written paper, with sufficient details (in main paper + supplement) that clearly introduce the problem, describe prior PF-based approaches, and build the proposed Mixture Density Particle Filter (MDPF) using a novel IWSG framework, following the ideas of Scibior et al., but using a kernel density-based (mixture density) resampler. + Addresses an important problem in the application of particle filter-based state estimators/trackers for models with unknown (parametric) dynamics and observation models (discriminative) + Propose two variants of MDPF, a baseline with a single mixture density used for both particle resampling and state estimation, and an improved A-MDPF, which employs different mixtures for the two tasks. A-MDPF is generally demonstrated to be more effective at the added computational cost of having two mixtures. + Extensive and convincing experiments conducted in simulated settings (both simple synthetic models and more complex 2D and 3D based simulators, c.f., Deepmaze and House3D with image based observations). Experiments demonstrate that IWSG leads to consistently more accurate state estimates compared to either prior works / baselines (LSTM, TG-PF, OT-PF, SR-PF, DIS-PF, C-PF, TG-MDPF, and IRG-MDPF), together with lower estimator variances.

Weaknesses

- Main paper does not clearly build the connection between Scibior et al. and the proposed approach, but this is clarified in the rather extensive supplement. The proposed approach can be seen as perhaps rather obvious, once the KDE mixture model is employed for the resampler, but it is nevertheless a fairly ingenious combination of MD + IWSG. - The authors acknowledge that PF-based approaches are not effective for high-dimensional state spaces and/or when the number of particles is large. However, they do not clearly investigate this dependency (all experiments are conducted on max 3D state spaces + bearing).

Questions

- How does the complexity of learning/inference depend on N? Do you have experimental evidence that demonstrates this? - Fig. 3 (supplement) shows some increasing variance trends in both gradients and states for MDPF (and A-MDPF). Can you comment on this / explain? Was this tendency observed in experiments in the simulator data?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

The authors has addressed limitations

Summary

This paper proposes an unbiased and low-variance gradient estimator for differentiable particle filters, where resampling steps incurs discrete changes that are non-differentiable in previous methods. The proposed method solved the problem by representing posteriors as continuous mixture densities, which is similar to the idea of regularized particle filters.

Strengths

The paper is well-presented, motivation and proposed method are clearly explained, experimental results demonstrated the effectiveness of the proposed gradient estimator for differentiable particle filters, especially when the posterior is multimodal.

Weaknesses

The performance of the proposed method when used in differnetiable particle filters trained with unsupervised loss when labelled data are not available is not discussed in the experiments/

Questions

1. As far as I can see, all the experiments presented in the paper assumed the true latent state is available during training, do you have any idea about how to train differentiable particle filters when true states are not available in training but we still want to track the latent state in testing? 2. The paper claimed the proposed estimator is unbiased, but it is not obvious to me why it is unbiased and I expect to see a theoretical proof of this.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

The proposed method is based on importance sampling, therefore some discussions on the variance of the estimator will improve the paper.

Response to author rebuttal

Thank you for the response (and apologies for my tardy response to the rebuttal). I appreciate the authors response to my concerns. I will maintain my score recommending acceptance.

Response to author rebuttal

Thanks for the authors' response, I will keep my score as it is now and recommend an acceptance.