Protected Test-Time Adaptation via Online Entropy Matching: A Betting Approach

We present a novel approach for test-time adaptation via online self-training, consisting of two components. First, we introduce a statistical framework that detects distribution shifts in the classifier's entropy values obtained on a stream of unlabeled samples. Second, we devise an online adaptation mechanism that utilizes the evidence of distribution shifts captured by the detection tool to dynamically update the classifier's parameters. The resulting adaptation process drives the distribution of test entropy values obtained from the self-trained classifier to match those of the source domain, building invariance to distribution shifts. This approach departs from the conventional self-training method, which focuses on minimizing the classifier's entropy. Our approach combines concepts in betting martingales and online learning to form a detection tool capable of quickly reacting to distribution shifts. We then reveal a tight relation between our adaptation scheme and optimal transport, which forms the basis of our novel self-supervised loss. Experimental results demonstrate that our approach improves test-time accuracy under distribution shifts while maintaining accuracy and calibration in their absence, outperforming leading entropy minimization methods across various scenarios.

Paper

Similar papers

Peer review

Reviewer 3rQG5/10 · confidence 4/52024-07-10

Summary

This paper proposes a novel test-time adaptation method based on martingales and online learning. It detects whether testing samples need to be adapted based on the sequential entropy values. Then, if a sample needs to be adapted, a pseudo-entropy value is computed for the adaptation. Overall, the idea of this paper is reasonable and interesting.

Strengths

1. The authors replace entropy minimization with entropy matching, which is interesting. Under this main idea, online drift detection and online model adaptation are naturally proposed and make sense. 2. This paper is well written and easy-to-follow.

Weaknesses

1. The experiments are relatively weak as the authors only conduct experiments on the ImageNet-C dataset, ignoring the CIFAR10-C and CIFAR100-C datasets. It would be better to present the results on datasets with a small number of classes. 2. The proposed method estimated the pseudo-entropy at testing time. However, I wonder whether this can be done when the label distribution [1, 2] also shifts because the shifted label distribution also affects the sequential entropy values. This should be discussed in detail, as well as the related papers. 3. The “Protected” in the title of this paper and name of this method should be carefully considered as the overall method seems not to explicitly ensure the safety of performance. [1] NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation. NeurIPS 2022 [2] ODS: Test-Time Adaptation in the Presence of Open-World Data Shift. ICML 2023

Questions

Please refer to the `Weaknesses` section. 1. The experiments show that the proposed method can achieve a lower ECE value. A further discussion about how these results can be done and their benefit for practice.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have discussed the limitations at the end of this paper.

Reviewer Nyvv6/10 · confidence 4/52024-07-11

Summary

This paper introduces a novel method for test-time domain adaptation using online self-training. It combines a statistical framework for detecting distribution shifts with an online adaptation mechanism to dynamically update the classifier's parameters. The approach, grounded in concepts from betting martingales and optimal transport, aligns test entropy values with those of the source domain, outperforming traditional entropy minimization methods. Experimental results demonstrate improved accuracy and calibration under distribution shifts.

Strengths

1. It is interesting to see betting and martingale appear in test-time adaptation, especially for modeling CDF for better shifted test sample prediction. 2. The paper is overall easy to follow, with rich experiments and visualizations. The algorithms clearly explain how the framework works. 3. The experiments on two TTA settings (single domain and continual TTA) confirm its effectiveness.

Weaknesses

1. It is not very intuitive to use betting in TTA. Although it might work for modeling CDF, martingale itself is not naturally suitable for TTA entropy CDF. 2. It is not very suitable to use the term "domain adaptation" in this context. Domain adaptation typically allows multiple epochs for adaptation, even in source-free settings, and uses target training samples while evaluating on target testing samples. In TTA, the same set of test data is used for adaptation and testing. 3. In Figure 1, the comparison with entropy minimization shows peaks, while entropy matching shows valleys when facing data in the tail for both classes, indicating they are both good indicators for class boundaries regardless of the changed data distribution. However, as described in lines 176-181, it seems the optimization will follow either the black line (entropy minimization) or the red line (entropy matching), making the meaning of this figure a bit vague. 4. The model requires calculating the source CDF, which requires extra time. Additionally, if there are source privacy concerns, it may not be possible to perform such calculations if the source model is only made available. 5. The experiments do not include an efficiency study, which is one of the motivations for doing TTA. 6. The paper does not include commonly used TTA datasets such as CIFAR10-C, CIFAR100-C, or any domain adaptation datasets such as OfficeHome, DomainNet, etc. 7. There is no sensitivity study. 8. This paper does not compare with state-of-the-art TTA baselines such as ROID [1]. [1] Marsden, R. A., Döbler, M., & Yang, B. (2024). Universal test-time adaptation through weight ensembling, diversity weighting, and prior correction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 2555-2565).

Questions

1. Can you explain how betting and martingales specifically contribute to the modeling of entropy CDF and why you believe this is a suitable approach for TTA? 2. In Figure 1, the comparison between entropy minimization and entropy matching is somewhat unclear. Can you elaborate on the intended interpretation of this figure and how it supports your claims about optimization following either the black or red lines? 3. The requirement to calculate the source CDF adds extra computational overhead and potential privacy concerns. How do you propose mitigating these issues, especially in scenarios where source model access is restricted or where computational resources are limited? ---

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

1. The paper uses the term "domain adaptation," which traditionally implies multiple epochs of adaptation and separate target training and testing samples. In the context of test-time adaptation (TTA), this terminology may cause confusion. A clearer distinction between these methodologies is suggested to avoid misinterpretation. 2. The requirement to calculate the source CDF introduces additional computational overhead, which has not been thoroughly discussed. In practical scenarios, especially where computational resources are limited or where source model access is restricted due to privacy concerns, this could pose significant challenges. An analysis of the method's computational efficiency and potential solutions to mitigate these issues would be beneficial. 3. The experiments conducted do not include widely recognized TTA datasets such as CIFAR10-C, CIFAR100-C, or domain adaptation datasets like OfficeHome and DomainNet. Including these datasets in the evaluation would provide a more comprehensive understanding of the method's generalizability and robustness. 4. The paper does not compare its results with state-of-the-art TTA baselines.

Reviewer Utu55/10 · confidence 2/52024-07-11

Summary

The paper addresses the problem of adapting a classifier to a new domain at test-time. It proposes a framework that first detects a distribution shift and, based on the detection results, adapts the classifier. The distribution shift detector employs a sequential test evaluating if the distribution of the test entropy deviates from the distribution of the source entropy. In the adaptation step, the normalization parameters of the model are updated via an entropy matching objective. Experiments are conducted on ImageNet-C.

Strengths

- The idea of using test martingales for test-time adaptation is novel to my knowledge and seems like a very natural application and interesting research direction. - The paper claims it only adapts when the distribution actually shifts, which is a desirable property of a TTA method, leveraging the trade-off between agile adaptation and keeping the source knowledge. - The paper is very well written overall. The reader is provided with intuitive explanations of the testing-by-betting framework - a very technical and not easy-to-explain theory - which makes the paper pleasant and insightful to read. - The experiments on ImageNet-C show encouraging results.

Weaknesses

**In short:** The paper proposes a very interesting approach, but it seems to me more work is required to round out the paper. In particular, the paper requires more empirical validation and clarifications. Some of the main weaknesses I see are: - The experimental results are quite limited. By showing results on only one dataset against three baselines, it is a bit unclear how the method performs across different datasets compared to existing methods. Comparing on other standard TTA benchmarks (such as CIFAR-10-C, CIFAR-100-C for corruptions, or Office-Home for domain adaptation) could help determine in which settings the method provides most gains and also its limitations. - The paper would benefit from an illustration visualizing the entropy matching procedure. In particular, it would be helpful to illustrate how $u_j, \tilde{u_j}, Z^t_j$, and $\tilde{Z}_j^t$ connect via the functions $F_s$ and $Q$. - More space and explanation could be dedicated to the actual entropy matching procedure. How can we match the two entropy distributions given $u_j$? This seems currently concentrated in lines 265-271 (see questions below).

Questions

- I’d appreciate some more clarification regarding section 3.4 (adaptation mechanism). Could you please elaborate on the role and interpretation of $Q$. I understood from lines 260-264 that $Q$ can be thought of as the distribution that approximates the unknown target’s entropy CDF, which makes sense to me given equations (3) and (6). However, in line 266 $Q$ seems to be used as a function to transform $u_j$ to $\tilde{u}_j$ by $\tilde{u_j} = Q(u_j)$. Could you explain the link between $Q$ being the target’s entropy CDF as well as a transformation? - I think the entropy matching could potentially be a good alternative to the dominant paradigm of entropy minimization, particularly since the latter has been shown to collapse eventually [1]. How does entropy matching perform on long-range adaptation? My understanding from the experiments is that only 1000 samples per corruption are used and not even the entire ImageNet-C dataset. [1] Press, Ori, et al. "The entropy enigma: Success and failure of entropy minimization." arXiv preprint arXiv:2405.05012 (2024).

Rating

5

Confidence

2

Soundness

2

Presentation

2

Contribution

3

Limitations

The limitations are rather vague, and it seems currently unclear which limitations the method encounters.

Reviewer Utu52024-08-08

Thanks for the rebuttal. I read all the responses and appreciate the additional clarifications. > CIFAR-10C and CIFAR-100C experiments / Office Home I appreciate the additional results, particularly including a domain adaptation dataset that provides more diversity in the distribution shifts tested. > Clarification on the entropy matching procedure. What is the role and interpretation of Q? How can we match the entropy distributions? Thanks for the clarifications. > How does entropy matching perform on long-range adaptation? My understanding from the experiments is that only 1000 samples per corruption are used and not even the entire ImageNet-C dataset. Thank you for detailing the lengths of the adaptation streams. I'm still wondering why the entire test set is not included in the experiments. For example, on CIFAR-10/100-C, the test set contains 10,000 samples per corruption type, and constructing a stream with 15 corruptions results in 150,000 test samples (instead of ~15,000 as used in Figure 1, rebuttal). To my knowledge, using all test samples is the standard evaluation setting (see e.g. [1]). Could you explain why you are diverging from the standard evaluation setting and subsampling? I understand you need a subset from the source domain for EATA and POEM, but this can be small and taken from the source data, right? Related to the above question, I am not sure if I am entirely convinced by the experimental evaluation. 1. If the focus of the paper is to propose a SOTA method, the evaluation against three baselines is too limited in my opinion. This concern of mine has not been addressed. 2. If instead the focus of the paper is to propose an alternative to entropy minimisation, I think three standard entropy minimisation methods as baselines seem sufficient. However, in this case, I am not entirely convinced of entropy matching being a robust alternative to entropy minimisation. In particular, I am not sure which of the cons of entropy minimisation does entropy matching address. The proposed approach seems to address the overconfidence issue on the source data, which the paper nicely shows does not occur with entropy matching. I think this is a promising result. However, I see one of the major limitation of entropy minimisation as that of model collapse after a long range of adaptation. However, it is unclear if this alternative addresses the said important limitation of entropy minimisation since the evaluated test streams seem to be even shorter then those in standard settings.

Authorsrebuttal2024-08-10

Follow-up to Reviewer Utu5 [1/2]

We thank the reviewer for their comments and for acknowledging our previous response. > I'm still wondering why the entire test set is not included in the experiments The test sets of ImageNet-C, CIFAR10-C, and CIFAR100-C consist of 15 different corrupted versions of the original test set of each dataset—these 15 corrupted versions represent different variations of the same “clean” test images. Therefore, we found it more natural to form an out-of-distribution test set that contains a single instance of a specific image, rather than using all 15 versions of the same original image. Additionally, we used shorter adaptation streams to demonstrate our approach’s ability to achieve faster adaptation compared to baseline methods, as shown in Figure 2 (bottom right). We apologize for any confusion and hope this explanation clarifies our initial choice. To address the reviewer's concern, we have now conducted experiments using the entire test set of CIFAR10-C and CIFAR100-C, which includes all 15 corrupted versions of each test image. The results are described hereafter. > I understand you need a subset from the source domain for EATA and POEM, but this can be small and taken from the source data, right? Yes, the holdout set can be small and should include unlabeled source samples. Since we use an off-the-shelf pre-trained model, we selected these holdout samples from the test set of the original dataset, representing the source domain. To ensure a fair out-of-distribution test set, we made sure our method (and EATA) does not have knowledge of the clean versions of the corrupted images in the test set. This is why we removed the holdout images from the corrupted test data, as the corrupted images are merely variations of the clean ones. **Long-range adaptation experiments on CIFAR-10C and CIFAR100-C:** In the following experiment, we reserved 2,500 images to form a holdout set for EATA and our method. We followed the same experimental protocol described in the global response to all reviewers and ran each adaptation method on a test set containing 112,500 samples (15 versions of 7,500 images). The results are summarized in two tables, presented in a separate comment below. These tables show that our proposed method is competitive with the baseline methods in terms of adaptation accuracy. Notably, the runtime of our method is twice as fast as SAR and comparable to EATA and TENT. Importantly, we **do not** use a model-reset mechanism (as done by SAR) or anti-forgetting loss (as done by EATA), highlighting the stability of our approach. By contrast, we found that TENT is highly sensitive to the choice of learning rate. We sincerely thank the reviewer for raising this point. > I am not entirely convinced of entropy matching being a robust alternative to entropy minimisation Beyond the theoretical aspects of our work and the monitoring capabilities we introduce, our experiments demonstrate several practical advantages over entropy minimization: 1. The proposed method maintains the performance of the source model while avoiding overconfident predictions under in-distribution settings, a crucial advantage over entropy minimization methods. 2. Short-term adaptation: Our approach achieves faster adaptation than entropy minimization methods, e.g., as shown in Figure 2 (bottom right). This is attributed to our betting scheme that quickly reacts to distribution shifts. While the reviewer emphasizes the problem of long-range adaptation, it is important to recognize the critical role of adaptation speed as well. Rapid adaptation is especially crucial, amid various strategies proposed for stabilizing long-range adaptation, such as resetting the self-trained model to its original state when specific heuristic conditions are met (as employed in SAR) or incorporating an anti-forgetting component into the entropy loss (as used in EATA). 3. Long-term adaptation: In extended test periods, our new experiments with 112,500 test examples show comparable adaptation performance to strong baseline methods, demonstrating the robustness of our method. Moreover, if stable long-range adaptation is the main concern, we could integrate model-resetting or anti-forgetting mechanisms. Notably, our monitoring tool can detect when unfamiliar corrupted data arrives, allowing for rigorous decisions on model resetting, for example, to prevent aggressive adaptation from a diverged state. This capability highlights another unique and practical advantage of our method. Once again, we apologize for any confusion and hope this discussion resolves the reviewer’s concerns. Please let us know if there are any questions, comments, or concerns left.

Reviewer Nyvv2024-08-11

Thank you for your detailed reply and additional experiments. I've also read the reviews and comments from other reviewers. I will increase the score.

Reviewer Utu52024-08-12

Thanks for the additional clarifications and experiments. I have raised my score. I'd encourage the authors to add the above clarifications on the experiments and the long-range adaptation results to the revised version.

Authorsrebuttal2024-08-12

Thank you!

Thank you for your engagement and for raising your score! We will certainly include additional clarifications on the experiments, the new experiments from the global response, and the long-range adaptation results in the revised version of the paper. We sincerely appreciate your thoughtful comments, which have helped improve our work.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC