NPCL: Neural Processes for Uncertainty-Aware Continual Learning

Continual learning (CL) aims to train deep neural networks efficiently on streaming data while limiting the forgetting caused by new tasks. However, learning transferable knowledge with less interference between tasks is difficult, and real-world deployment of CL models is limited by their inability to measure predictive uncertainties. To address these issues, we propose handling CL tasks with neural processes (NPs), a class of meta-learners that encode different tasks into probabilistic distributions over functions all while providing reliable uncertainty estimates. Specifically, we propose an NP-based CL approach (NPCL) with task-specific modules arranged in a hierarchical latent variable model. We tailor regularizers on the learned latent distributions to alleviate forgetting. The uncertainty estimation capabilities of the NPCL can also be used to handle the task head/module inference challenge in CL. Our experiments show that the NPCL outperforms previous CL approaches. We validate the effectiveness of uncertainty estimation in the NPCL for identifying novel data and evaluating instance-level model confidence. Code is available at \url{https://github.com/srvCodes/NPCL}.

Paper

References (55)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer 1ufQ4/10 · confidence 4/52023-06-30

Summary

This paper proposes a new method for Continual Learning (CL) through Neural Processes (NPs) framework. Especially, they are inspired by MTP [1] which utilizes the global latent and task-specific latent for inter-task and intra-task knowledge representation. In this paper, the authors additionally regularize on parameter updates of MTP not too much be updated, and task-wise inference through an uncertainty quantification metric. Through this, they can achieve comparable performance with state-of-the-art prior works while maintaining smaller memory. [1] Buzzega, Pietro, et al. "Dark experience for general continual learning: a strong, simple baseline." Advances in neural information processing systems 33 (2020): 15920-15930.

Strengths

- This paper proposes new regulations to utilize NPs for CL. - The prior works are well summarized and they discussed their model adequately. - They evaluated it on multiple benchmarks including class and domain incremental learning benchmarks with diverse prior works including NPs [1] and ANPs [2], and showed their model, the NPCL shows comparable performance to state-of-the-art methods such as DER [3]. - They studied diverse ablations such as their regularization losses, latent types (e.g., when only task-wise latent is learned), task-wise inference, and context sizes. In this study, they showed their task-wise inference is important for their model's performance and the uncertainty on the logits of the task heads is a good measurement for task label estimation. Additionally, they also compared their model with DER for the storage overhead. For this aspect, NPCL shows more efficient storage utilization than DER. - They also showed the uncertainty on the task heads can be utilized to identify the novel data from out-of-distribution or not and uncertainty-based task-wise inference can show higher confidence for correct task labels. - Lastly, they clearly discussed the limitations of their model such as more computational overhead and inference time complexity caused by multi-head attention. [1] Garnelo, M., Schwarz, J., Rosenbaum, D., Viola, F., Rezende, D. J., Eslami, S. M., & Teh, Y. W. (2018). Neural processes. arXiv preprint arXiv:1807.01622. [2] Kim, H., Mnih, A., Schwarz, J., Garnelo, M., Eslami, A., Rosenbaum, D., ... & Teh, Y. W. (2019). Attentive neural processes. arXiv preprint arXiv:1901.05761. [3] Buzzega, P., Boschini, M., Porrello, A., Abati, D., & Calderara, S. (2020). Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 33, 15920-15930.

Weaknesses

- The motivation is not strong enough. Their motivation to utilize NPs for CL is to measure predictive uncertainties. However, it is not clearly discussed why it is important for CL. - Figures (Figures 1, 2, 4, and 6) are not recognizable. Especially their fonts are too small. For Figure 6, they missed labeling the axes. - In lines 38-39, they mentioned one of their motivations is NPs can meta-learn input correlations across correlated tasks. However, in the limitation section, they analyzed it as the incompetence of dot-product attention (lines 305-306). It is the opposite analysis of their motivation. - The equation (7) and relevant explanations are confusing. In the previous section for GR, task $t$ is dominated by the t-$th$ task samples, while it is referred to as $j$ in this section. As I understood, they tried to regularize the task $t$'s task latent on that at the step when the task $t$ arrived, but it is not easy to follow in their explanation. - The equation (8) is hard to understand. In their explanation about the memory management of their model, the sample $x$ is not stored, but it is not in this equation (without $x$, $L_{CE}$ cannot be calculated). - The comparison in Table 1 can cause confusion. At first glance, we can expect NPCL and DER to show comparable performance in the same memory usage, but their storage usages are quite different as shown in Appendix E.3. I can recommend showing as a plot in which the x-axis is real storage usage and the y-axis is accuracy. - (minor) typos - In equation (8), the bracket for $L_{GR}$ and $L_{TR}$ is missed. - In table 2, baseline is not w/o GR *or* TR, but GR *and* TR. - In line 208, with. -> with

Questions

- Can you additionally analyze why small $N$ is not working well with the additional experimental results for diverse $N$ values? - In your design, the sum pooling is used to get the global and task-specific latent. Could you replace it with Multi-head Attention Pooling (MAP) [1] (e.g., using the learnable additional token for the latent) and test it for a few tasks? [1] Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., & Teh, Y. W. (2019, May). Set transformer: A framework for attention-based permutation-invariant neural networks. In International conference on machine learning (pp. 3744-3753). PMLR.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

The authors adequately discussed the limitations of their work. Thanks for their detailed analysis of their limitations.

Reviewer v3Yx6/10 · confidence 4/52023-07-05

Summary

This paper suggests tackling continual learning (CL) with a neural process model (NP). The authors describe a hierarchical latent variable model, with one global latent, and t per-task latents. As is common in NPs, the latent posteriors are a function of a context of datapoints with their corresponding labels, and then a decoder maps the latents to an output prediction. During training time, the model is trained with access to task labels, and uses data from different tasks to train the corresponding task-specific latent posteriors. A replay buffer stores and reuses data from old tasks during training, and additional regularization terms are introduced, in order to minimize effects of drifting away from previous tasks ('forgetting'). At test time, there is no access to task labels, and therefore the model is run for all tasks, and the output with the least entropy is used. The model is compared to various CL approaches and is shown to achieve and sometimes surpass the state of the art. The paper describes more ablations and analysis of the results.

Strengths

The main strengths of this paper are: 1. An interesting model with non-trivial components put together: - the hierarchical structure of the latents - the difference treatment of training time and test time with respect to access to context latent 2. The result seems to surpass previous approaches 3. Extensive analysis of the results

Weaknesses

The main weakness of this paper is that modeling continual learning as meta learning via neural processes is not exactly valid. Specifically, NPs assume that the dataset contains many different functions or tasks, and one of the main challenges is that the task id is not accessible and has to be inferred. This is the reason a "context" of labeled points is used - to infer the task at hand. In the setup described in the paper, it is assumed that the task id is accessible at training time, which allows designating different parts of the model to different tasks. At test time, the model uses a context coming from the training data with little overlap to the task of the test data. This is a major difference from NPs. I think that the presented model is interesting and can have a significant contribution to the community but that the good performance exhibited is not connected to NPs as stated, but rather merely a result of the higher capacity of hierarchical latent variable models. Since the model is conditioned on training points during evaluation, perhaps a better way to present it is as a non-parametric method (e.g.[1]) rather than a neural process. [1] Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning. Jannik Kossen, Neil Band, Clare Lyle, Aidan N. Gomez, Tom Rainforth, Yarin Gal

Questions

1. The paragraph in line 110 is not clear. What does it mean to optimize parameters of $\mathcal{D}^t$? 2. In equation 3, does the context define the task (as modeled in standard NPs)? If so why are there superscripts of all the tasks 0:t? 3. In equation 4 why are $z^t$ conditioned on the contexts $\mathcal{C}^t$? If the task id is given then the context doesn’t have any information. 4. The number of samples described in line 151 is not clear. If every point is mapped to one task then only the corresponding latent needs to be sampled, making the number of samples M. Is this wrong?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

I did not find any unaddressed limitations or potential negative impact in this work.

Reviewer fvgr7/10 · confidence 4/52023-07-07

Summary

This paper presents an uncertainty-aware continual learning framework that utilizes Neural Processes (NPs). The NP model employs a hierarchical latent variable model in conjunction with an experience replay buffer, where a global latent variable captures inter-task correlations and task-specific latents encode more detailed knowledge. To prevent catastrophic forgetting, the method regularizes a Jenson-Shannon divergence between current and past distributions of latent variables. The paper employs entropy for uncertainty quantification.

Strengths

- The concept of regularizing latent distributions between preceding and current tasks is both simple and intuitive. - Experimental results demonstrate that the proposed approach is competitive with state-of-the-art continual learning methods, while also being capable of quantifying model uncertainty. - The proposed method only stores two vectors for each task, showcasing memory efficiency in terms of experience replay. - Generally, the paper is well-written and easy to understand.

Weaknesses

- Given that the encoder includes self-attention and cross-attention layers, the proposed method exhibits quadratic complexity relative to the training data. This complexity could prove problematic when the training dataset is substantial (e.g., 10000).

Questions

- Why was Jensen-Shannon Divergence (JSD) chosen as a regularization metric? Have other distribution divergence metrics, such as Kullback-Leibler Divergence (KLD), been explored? - A key strength of the NP family is its few-shot learning capability. Given that the size of the replay buffer can be a limiting factor in some applications, would the proposed approach still be effective with a much smaller buffer size (e.g., 10)? - How do accuracy and uncertainty depend on the size of the Monte Carlo sampling (N, M)?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors have addressed the limitations.

Reviewer qdzJ6/10 · confidence 3/52023-07-07

Summary

This paper introduces an uncertainty-aware continual learning framework based on neural processes. The proposed method casts CL into a hierarchical latent model from the global variables to the task-specific variables, and the corresponding regularization terms for each to ensure minimum forgetting along the course of new task learning. The naturally reliable uncertain estimation of NP facilitates class incremental setting by allowing accurate task-specific latent selections.

Strengths

- The presentation of this paper is mostly clear, with the equations properly delivering the ideas. - The idea of applying NP in CL, explicitly modeling task-specific latent variables, and uncertainty quantification for class-incremental CL are very intriguing.

Weaknesses

**Writing and clarity** The writing can be further improved. For example, in my understanding, following Line 163, the two regularization terms are introduced to counter the forgetting instead of being 'two key aspects of forgetting in NPCL.' In line 114, the authors mentioned 'uses generative factors', while the term 'generative factors' is not appropriately explained here or in the following sections. **Quantitative comparisons** While clear improvements over some baseline methods and other NP-based methods are reported, I believe the compared CL methods are all relatively classic but old. More up-to-date might be helpful to better position the proposed method. Other minor points: - Figure 2 is too small - The authors might consider refining the formats of the reference list, as currently the formats are very inconsistent.

Questions

Please see the Weakness section. How is the uncertainty estimation of the proposed method quantitatively? Some further evaluation using metrics like ECE can be more insightful.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors discussed the limitations of the proposed method very comprehensively in Section 6.

Reviewer 1ufQ2023-08-14

Reply to the response of the author

Thank you for your response to my concerns. I think they properly discussed them. +Can you update the figures? They are hard to understand due to very small fonts and missed x and y axis labels.

Authorsrebuttal2023-08-14

Dear Reviewer 1ufQ, Thank you for taking the time to review our rebuttal. We have considered your feedback regarding the figures and plan to update our manuscript at the earliest allowed revision. Let us know if you have any other questions or concerns. Also, if you are satisfied with our answers, please consider revising your score. With best regards

Authorsrebuttal2023-08-17

Dear Reviewer 1ufQ, We have put our best efforts in attempting to address most of your concerns by providing more reasonable explanations and/or new experimental results. If our response addresses your concerns and clarifies the value of our paper, we'd be truly grateful if you could reconsider your recommendation. Please be assured that we are open and eager to continue the discussion should there be any aspects of our paper that remain unclear to you. Thank you once more for taking the time to engage with our response! With best regards

Authorsrebuttal2023-08-15

Dear Reviewer qdzJ, We thank you again for taking your time reviewing this work. We put our best efforts to prepare the rebuttal to your questions. We would very much appreciate if you could engage with us with your feedback on our rebuttal. We would be glad to answer any further questions and clarify any concerns. Also, if you are satisfied with our answers, please consider revising your score. With best regards

Authorsrebuttal2023-08-15

Dear Reviewer v3Yx, We thank you again for taking your time reviewing this work. We put our best efforts to prepare the rebuttal to your questions. We would very much appreciate if you could engage with us with your feedback on our rebuttal. We would be glad to answer any further questions and clarify any concerns. Also, if you are satisfied with our answers, please consider revising your score. With best regards

Reviewer v3Yx2023-08-15

Thank you for your answer. I'm still not sure I understand how the problem is modeled as a NP. In NPs the dataset is considered to be a dataset of different functions X->Y, where each sample is a specific function. NPs are trained as generative models of these functions in a way that enables flexible conditioning. So given a trained NP, we can generate a complete function given a context set of observed x,y points along the function, or even generate a new function without any observed (context) points. This formulation can be mapped to the few-shot learning setup (i.e. meta-learning), where each task is a function x->y, and again we have a dataset of those tasks (functions) and the few shot examples available for each task serve as the context of x,y pairs from the specific function. How is this formulation mapped to the continual learning setup? Does the equivalence between 'task' and 'function' hold here? From my understanding, in [1], the 'tasks' are different than the 'functions'. One way to think about it is that each function has a multi-dimension output y which corresponds to the different tasks. Does this formulation apply to your model as well?

Authorsrebuttal2023-08-16

Further clarification on the problem setting

Dear Reviewer v3Yx, Thank you for the comment. We agree with you on NPs for the meta-learning (ML) settings including few shot problems. To clarify the problem setting and your further questions, we have prepared the following three points whilst being more specific in terms of notations. **A. ML vs CL (ours) setting:** In both ML and continual learning (CL) settings, we have multiple tasks and we would like to learn an NP to generate task-specific functions, which aligns with “NPs are trained as generative models of these functions in a way that enables flexible conditioning”. Specifically, for each task t, we have $Y = f_{Z_t}(X)$ and $Z_t$ is the latent variable conditioned on the task-specific contexts. As $Z_t$ is task-specific, this makes $f_{Z_t}$ also task-specific. There are two major differences between standard NPs for ML and our proposed NP for CL: 1. In standard NPs, $Z_t$ is conditioned on $(X^c_t, Y^c_t)$, which are context samples of task $t$. While in our NP, $Z_t$ is conditioned on $(X^c_t, Y^c_t, Z_G)$. $Z_G$ is from $X^c_G, Y^c_G$ which are the data samples from all the tasks. To accommodate the properties of CL better, we introduce a hierarchical model that introduces more sharing in $f_{Z_t}$ across the tasks via $Z_G$. 2. The goals of ML and CL are quite different. ML aims to learn NPs that can generalize to new tasks. That is why NPs are tested in a new task $t*$ given its context samples $(X^c_t*, Y^c_t*)$. In CL, our goal is to have a good $f$ that can perform well on all the seen tasks. Therefore, there is no new task during inference. Similar to other works adapting NPs to learn such a good $f$ generalizing all the seen training tasks [1], our context samples during testing are a subset of the training data for the tasks seen so far. To summarize, CL requires to learn a general function $f$ that is task-specific but also performs well on all the seen tasks given a limited rehearsal memory. To achieve this, we leverage NP ability of generating task-specific functions with flexible conditioning. But instead of using the standard model of NPs, we propose a hierarchical model to introduce more sharing, tailored for the CL problem. **B. Task-to-function equivalence:** A CL task may or may not involve data points sampled from a single function $f_{Z_t}: x$->$y$. For a standard CL setup (like ours), the equivalence between a task and a function does hold since each data point $x$ in a task is mapped to a single label $y$. However, if we were to adapt NPs for a multi-label CL setting [3,4], then similar to [2], at each (incremental) training task, we would learn a function with a multi-dimensional output $y$. You are thus correct that the tasks and functions are not equivalent in [2]. But for our CL setup, where each $x$ has a single label $y$, a task–to–function equivalence holds. In fact, we note the difference in the nature of prediction variables $y$ between ours and [2] in the related work **lines 84-85 (see point (b))** in the main paper. Based on this discussion, we will rephrase the line to make it more explicit regarding the equivalence. **C. Limited context scenario:** Finally, just like the standard NPs, NPCL does not need a complete set of context points over all the learned functions to be able to infer these although a diverse set of context does certainly help. This is evident from our ablation on the varying context sizes during inference (lines 253-256 in main paper). Here, a context set size of as little as 5 still helps us maintain a competitive accuracy on S-CIFAR-10. We hope the above points help clarify your doubts regarding the usage of NPs in our problem setting. Please let us know if you have any further questions or concerns. We look forward to your feedback. *References*: [1] Wang, J., Lukasiewicz, T., Massiceti, D., Hu, X., Pavlovic, V., & Neophytou, A. (2022). NP-Match: When Neural Processes meet Semi-Supervised Learning. ICML. [2] Kim, D., Cho, S., Lee, W., & Hong, S. (2022). Multi-Task Processes. ICLR. [3] Liang, Y., & Li, W. (2022). Optimizing Class Distribution in Memory for Multi-Label Online Continual Learning. ArXiv, abs/2209.11469. [4] Du, K., Lyu, F., Li, L., Hu, F., Feng, W., Xu, F., Xi, X., & Cheng, H. (2022). Multi-Label Continual Learning using Augmented Graph Convolutional Network. ArXiv, abs/2211.14763.

Reviewer v3Yx2023-08-16

Thanks, this is a bit clearer now, but I'm still not sure I understand the roles of $Z_G$ and the different $Z_t$'s. Could you please describe the way the model is used at test time? More specifically, given a new target $x$ coming from an unknown task $t^*$ 1. What context is used to infer $Z_G$? 2. What context is used to infer the different $Z_t$'s? 3. Are the different $Z_t$'s inferred with the same encoder network? (i.e. are the weights shared). 4. How is the single predicted y computed?

Authorsrebuttal2023-08-16

Inference using NPCL

Dear reviewer v3Yx, We are happy to have made our problem setting more clear to you. Here is a brief summary of how NPCL performs task-agnostic CL inference for a given test sample $x*$. At test time, we first make use of the training samples stored in the CL rehearsal memory $\mathcal{M}$ as our context samples. Note that in addition to labels $y$ for the data points $x$, our setting also stores the task ID $t$ for each $x$ in the memory $\mathcal{M}$. The availability of the latter task IDs help us derive the task-specific latents $Z_t$’s at test time similar to how we derive them during training. In what follows,for brevity, we conceal the role of the feature extractor $f$ (e.g., a ResNet-18) used to derive the features of a data point $x$ before feeding these to the NPCL. Let us say we are performing the inference after having seen $(t+1)$ tasks, i.e., we have trained a total of $(t+1)$ task-specific encoder modules of NPCL so far. The NPCL inference then proceeds with the following steps: - **Shared global encoder:** The context samples from $\mathcal{M}$ are used to derive the $N$ Monte Carlo (MC) samples of global { ${Z_G}_{i=1}^N$ using the global encoder module. **Note that the global encoder is shared by all tasks.** - **Individual task encoders and their respective context:** For deriving $Z_t$'s, we rely on the task ID $t$ for each context sample $x \in \mathcal{M}$. As shown in Fig. 2 in the main paper, the input to each task-specific encoder are the intra-task attended features $s^t$ concatenated with the $N$ global MC samples ${Z_G}$. Following this, we sample $M$ task-specific latents {$Z_t$}$_{i=1}^M$ from the $(t+1)$ task encoders. Note that we have the access to the task ID $t$ for the memory samples (which as stated above, are simply a subset of the training samples used in a typical experience replay CL setup). **Also, note that the weights for the $t-$th task-specific encoder is shared only among the context samples belonging to the $t-$th task.** Therefore, these task IDs can be used to redirect the features of the context samples to their respective task encoders. At the end of this step, we have a total of $(t+1) \times N \times M$ task-specific latents that we can possibly employ for decoding. - **Shared decoder:** Now, for a given task sample $x*$, we are left with $(t+1) \times N \times M$ number of possible task-specific latent $Z_t$ to infer from. What we know is that out of these $(t+1) \times N \times M$ task-specific latent samples, one set of $ N \times M$ latent samples belongs to the correct task encoder for $x*$, i.e., conditioning the decoding of $x*$ with this set of $ N \times M$ latent samples should give us the most accurate prediction $y*$. To identify this correct set, we first decode predictions for $x*$ conditioned on the entire set of $(t+1) \times N \times M$ of $Z_t$'s. In particular, we feed the concatenated feature [$x*; z_t$] to the decoder. - **Uncertainty-based inference:** The decoder module outputs a total of $(t+1) \times N \times M$ logits corresponding to each $Z_t$. Next, for each set of $ N \times M$ logits (corresponding to the individual task encoders), we compute the Shannon entropy scores using eq. (9) in the paper. We then consider the set of logits with the lowest Shannon entropy as the final predictions. - **Single y** * **prediction:** To get the single final prediction $y*$, we compute the average of the above set of $ N \times M$ logits. We hope the above steps make the roles of $Z_G$ and $Z_t$’s more clear to you. We would be happy to answer any further questions. We look forward to your response. With best regards

Authorsrebuttal2023-08-21

Dear Reviewer v3Yx, As we approach the end of the discussion period, we are thankful to you for your precious time and engagement. In particular, we were pleased to hear that our response made the validity of NPs in our setting more clear to you. We would thusly be grateful if you could reconsider your recommendation based on our follow-up discussion. Thank you again for your consideration. With best regards

Reviewer v3Yx2023-08-22

After the above discussion I accept that the presented model is a valid formulation of NPs. Since this addresses the main concern I had, I am raising my score.

Authorsrebuttal2023-08-15

Dear Reviewer fvgr, We thank you again for taking your time reviewing this work. We put our best efforts to prepare the rebuttal to your questions. We would very much appreciate if you could engage with us with your feedback on our rebuttal. We would be glad to answer any further questions and clarify any concerns. With best regards

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC