Category-Extensible Out-of-Distribution Detection via Hierarchical Context Descriptions

The key to OOD detection has two aspects: generalized feature representation and precise category description. Recently, vision-language models such as CLIP provide significant advances in both two issues, but constructing precise category descriptions is still in its infancy due to the absence of unseen categories. This work introduces two hierarchical contexts, namely perceptual context and spurious context, to carefully describe the precise category boundary through automatic prompt tuning. Specifically, perceptual contexts perceive the inter-category difference (e.g., cats vs apples) for current classification tasks, while spurious contexts further identify spurious (similar but exactly not) OOD samples for every single category (e.g., cats vs panthers, apples vs peaches). The two contexts hierarchically construct the precise description for a certain category, which is, first roughly classifying a sample to the predicted category and then delicately identifying whether it is truly an ID sample or actually OOD. Moreover, the precise descriptions for those categories within the vision-language framework present a novel application: CATegory-EXtensible OOD detection (CATEX). One can efficiently extend the set of recognizable categories by simply merging the hierarchical contexts learned under different sub-task settings. And extensive experiments are conducted to demonstrate CATEX's effectiveness, robustness, and category-extensibility. For instance, CATEX consistently surpasses the rivals by a large margin with several protocols on the challenging ImageNet-1K dataset. In addition, we offer new insights on how to efficiently scale up the prompt engineering in vision-language models to recognize thousands of object categories, as well as how to incorporate large language models (like GPT-3) to boost zero-shot applications. Code is publicly available at https://github.com/alibaba/catex.

Paper

Similar papers

Peer review

Reviewer 194m6/10 · confidence 3/52023-06-19

Summary

The paper presents a new technique to improve OOD, as well as IID, prediction for a pre-trained Language Vision Model (LVM). The heart of the proposed method is to learn both an ID context (i.e. perceptual context) and an OOD context (i.e. spurious context) to improve the classification of both ID classes and OOD samples. The paper proposes a new loss function to combine the ID and OOD losses together and a sampling strategy to produce spurious samples.

Strengths

The strengths of the paper are in its empirical validation and the originality around combining insights from different works. The proposed technique in the paper shows strong empirical performance, including ablation testing, on the main OOD tasks and measures. The quality of this validation well supports the claims made in the article about the need to consider both ID and some kind of spurious context when doing open-world detection. The paper also combines elements from previous work, like the idea of having perturbed examples from VOS and NPOS, and the idea of learning prompts in the text vector space from Learning to Prompt into one framework.

Weaknesses

Despite its strong empirical validation, the paper does have some weaknesses in its clarity and novelty. Beginning with novelty, the proposed technique of CATEX seems to be only an incremental improvement on NPOS (e.g., creating perturbed examples as part of training for OOD) and directly uses the technique from Learning to Prompt with only a change in the loss function. In essence, the paper doesn’t present any insight that the NPOS/VOS papers already presented, namely the inclusion of perturbed samples into learning for an LVM can help with OOD performance. If the paper were more explicit, especially in the methods section and discussion section, about how the proposed method differs from previous ones, it would help for establishing the novelty. For example, I believe both VOS and NPOS train the underlying CLIP model, while the proposed technique of CATEX uses the Learning to Prompt technique of training a lightweight layer on top of the CLIP model. Such a change seems to balance between being good at ID tasks, while not distorting the feature space. In terms of clarity, there is not enough detail in the methods section to both deal with the aforementioned novelty issues as well as to fully understand the training process and the perturbation guidance. For the perturbation guidance, lines 174-176 make it sound like its changing the actual words or tokens (as is done in the Kwon et al. article with masked language modeling), versus the embeddings of the text, as is done in Learning to Prompt article. If the proposed technique is actually masking the tokens, versus changing the context vectors in the embed space, then how is the training done to optimize the text, as the method in Learning to Prompt only deals with a vector space? Also, does perturbation by masking tokens fully make sense? For example, is the perturbation of “a photo of a dog” to “a [MASK] of a dog” or “a photo of a [MASK]” really a meaningful perturbation for the spurious context? I wish there was something like a walkthrough example of the perturbation as well as some more explanation of the training method and intuition behind the perturbation to better understand the contribution of the work. Finally, there are a couple of areas where the writing could be improved. There are some sentences throughout that need to be proofread for grammar. For example, the last sentence of the abstract is a run-on sentence and the sentence on lines 48-50 is unclear in what its trying to say. -------- Following author's responses -------------- I believe the authors have significantly addressed my concerns about the perturbation guidance. They have both added additional explanations as well as done some additional experimentation around ideas like how many tokens to mask.

Questions

1. How does the proposed method find optimal perturbations of the textual input? How does the method decide on which tokens or words to mask? 2. How could the proposed method be used when you have no labeled data (i.e. a true zero-shot setting) to improve performance? Or can it? For example, can perturbations be included in at inference time – combined with the OOD scoring function – to do zero-shot labeling? 3. What is the performance of the proposed method versus CoCoOP? While the paper does investigate the performance of its proposed method versus CoOp, and rightly concludes its does better with OOD, it does not evaluate the performance of its proposed method against the newer version of the CoOP method (i.e. CoCoOP) which was explicitly designed to deal with OOD issues of CoOP. -------- Following author's responses -------------- I believe the authors have answered all three of my questions. In particular, I found their answer around 2, of how to use the proposed method to improve the zero-shot performance of LVMS, to be very interesting and notable. I also find it quite interesting that there was such a performance gap between the proposed method and CoCoOP.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

The authors have addressed nearly all of their limitations and addressed those dealing with societal impact. The only limitation they have not addressed is that the proposed method still requires labeled (or captioned) data in order to work and cannot work in a zero-shot setting as was the promise of CLIP. I welcome the author’s reply on this, as I am not sure if the proposed method couldn’t be used without labeled data.

Reviewer R97w6/10 · confidence 3/52023-07-04

Summary

The paper proposes a method to incorporate perceptual context and spurious context to handle the OOD detection problem. The experimental results seem quite promising.

Strengths

1. The results of the proposed method is obviously better than previous OOD detection methods; 2. The proposed method is quite interesting, which i think can be extended to other areas involving large-scale vision-and-language models.

Weaknesses

1. The presentation of this paper is not very clear. For example, the authors mentioned "label bias" in the contributions part, but there is no explicit explanation about this so-called label bias, which I believe needs more clarifications; 2. The motivations for the proposed hierarchical context mechanism is not very clear. Why a single context cannot produce a precise classification boundary? Moreover, the spurious samples w.r.t. a specific ID category can have large variance, so using a single spurious context can encode such large intra-class variance? 3. It would be better if the authors can provide some visualizations of the generated samples using the perturbation guidance.

Questions

Kindly refer to the weakness part

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

The authors can test that finetuning the encoders from the CLIP can produce what extent of the performance improvement for OOD detection.

Reviewer oU436/10 · confidence 5/52023-07-06

Summary

This paper contributes a new method for OOD detection by learning the precise category boundaries, specifically, the category boundaries are defined by one perceptual context and one spurious context, these two contexts are learned text embeddings for a frozen CLIP model, the spurious context is learned by perturbing the perceptual context. Experiments on large scale OOD detection benchmarks show the effectiveness of the proposed method.

Strengths

1. The experiments are comprehensive. 2. Because the proposed method does not change the parameter the frozen CLIP model, it is shown that the proposed method is more generalizable under multiple category extended scenarios. 3. The writing of the paper is clear.

Weaknesses

1. The spurious context is learned by perturbing the perceptual context, and the perturbation is done by changing one word embedding, this could also be changing multiple word embeddings, but there are no ablations for this. 2. There is one paper[R1] explore similar idea of spuriour context (termed reciprocal points in [R1]), which I think should be cited and discussed. [R1] Learning Open Set Network with Discriminative Reciprocal Points, ECCV 2020.

Questions

1. I don't quite understand why the perceptual context is called hierarchical.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The limitations have been discussed in the paper.

Reviewer rNwN6/10 · confidence 4/52023-07-07

Summary

This paper proposes a framework for detecting out-of-distribution (OOD) samples using hierarchical description - perceptual and spurious contexts. Authors consider a category-extensible setup where categories can merge hierarchically. The proposed approach is evaluated by considering ImageNet as the in-distribution data and iNaturalist, SUN, Places, and Texture as OOD datasets.

Strengths

1. Considering image-text models to learn precise class boundaries is interesting and effective. 2. Authors performed extensive experiments on relevant benchmarks. 3. Results are state of the art.

Weaknesses

1. It is not clear from the introduction what the hierarchical contexts are and how they affect OOD detection in a category-extensible way. Precisely, how do these contexts influence Fig. 1(a) to get updated to Fig. 1(b)? 2. Authors talk about learning precise class boundaries. This is perhaps more suitable for novel class detection i.e., open-set learning than OOD detection. For example, the same class 'Car' can be in-distribution in a sunny environment and OOD in a rainy environment. How does the proposed approach address this scenario? 3. Line 150-152: how does the combination of the contexts implemented? It is not clear from the description. 4. section 3.2 and Fig. 3: how are the spurious samples generated and perturbed? Are the perturbations always random or informed by text cues?

Questions

Please address the questions in the 'Weaknesses' section.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Yes

Reviewer ufZG6/10 · confidence 3/52023-07-10

Summary

This paper presents a solution for the task of out-of-distribution detection with a hierarchical context. Specifically, it introduces the concepts of spurious context as negative descriptions to learn the distribution of unseen categories implicitly. When equipped with perceptual context, it can effectively detect OOD samples. Experimental results verify its robustness and effectiveness on several benchmarks.

Strengths

- The proposed method is simple yet effective. Instead of finding a negative context for all categories, it proposes to adopt one for each category. Besides, the approach proposed to find negative samples is novel and intuitive. - The writing structure is clear and easy to follow. However, I find it hard to have an initial guess for the meanings of "perceptual context" and "spurious" context in the abstract section. - The experimental results are strong and solid. Besides, the ablation study in Tab.4 verifies the effectiveness of both proposed modules.

Weaknesses

- The masking ratio in section 3.2, as a very important factor, hasn't been studied. A too-small ratio may lead to some false negative candidates while too-large ratios can neglect false positive candidates. From this perspective, this hyper-parameter can be very sensitive, thus influencing performance vastly. - I just doubt whether a single vector w^p_k for a spurious can model the complex category boundary. Modeling the interior region of a category is intuitive as in the ideal case it can model a quasi-hypersphere. But for OOD samples surrounding a category, the manifold can be more complex. - Fig.4 (a-b) as the main visualization, is a bit confusing. It's hard to tell what it aims to show.

Questions

What is the masking ratio in Pertubantion Guidance? And is the performance sensitive to it? Where will the spurious context locate? How about plotting them in Fig 4(a-b)? Although there has been an illustrative plotting in Fig.1, what would it be like in TSNE visualization?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

No potential negative societal impact

Reviewer R97w2023-08-11

The author's response fully addresses my concerns and the added experiments further demonstrate the value of this work. So I decided to raise my rating to weak accept.

Reviewer 194m2023-08-13

Reponse to Author's Rebuttal

I really appreciate the authors' addressing of my questions and attempting to better elucidate how the perturbation is done. In particular, I am rather impressed by the performance of the proposed method against CoCoOP (which is supposed to handle OOD better) and how the method can be used to improve zero-shot image classification. This later answer, in particular, raised my estimation of this paper rather significantly. I do still wish there was some kind of easier-to-follow flow chart or even pseudo-code on how to implement the perturbation of CATEX for use in training and zero-shot classification, though. Given the authors' responses, I have raised my rating and do believe the paper should be given serious consideration for acceptance.

Authorsrebuttal2023-08-14

We thank the reviewer again for evaluating our work and carefully reading our response. Your constructive comments and insightful suggestions do make our work much stronger, and we are really encouraged that you can boost our paper. On the other hand, we are sorry that in the discussion stage, it is not allowed to provide new charts or figures to illustrate our method. Therefore, to answer your following-up questions on how our perturbation is implemented in (1) training our CATEX and (2) performing zero-shot classification, we provide the pseudo-codes below: > 0. How is the perturbation implemented? The perturbation itself is simple. For a context $\mathbf{v}$, the perturbation can be expressed as: 1. Take the $m$ (e.g., 16) learned/pre-defined tokens/words $\mathbf{v} = [v_1;v_2;\cdots;v_m]$ 2. Generate masking/noise perturbations $u$ 3. Randomly perturb one (or more) token/word $\mathring{\mathbf{v}} = [v_1;u;\cdots;v_m]$ The above process is formulated as $\mathring{\mathbf{v}} = \mathcal{P}(\mathbf{v})$. With the text-encoder $\mathcal{T}$ and class name $[\texttt{CLS}]$, the perturbed text embedding is encoded as $\mathring{\mathbf{w}} = \mathcal{T}(\mathring{\mathbf{v}}, [\texttt{CLS}])$. > 1. How is the perturbation implemented in the vanilla training procedure? Our method trains a pair of perceptual context $\mathbf{v}^p$ and spurious context $\mathbf{v}^s$ for each category, with real in-distribution samples $\lbrace\mathbf{x_i}\rbrace$ and generated OOD samples (guided by the perturbation) $\lbrace\tilde{\mathbf{z}}_j\rbrace$. The training procedure can be expressed as: 1. Perturb the perceptual context $\mathring{\mathbf{v}}^p = \mathcal{P}(\mathbf{v}^p)$ 2. Generate random OOD samples $\lbrace\tilde{\mathbf{z}}_j^\prime\rbrace = \mathcal{G}(\lbrace\mathbf{x_i}\rbrace)$ 3. Use perturbation to select OOD samples $\lbrace\tilde{\mathbf{z}}_j\rbrace = \mathcal{F}(\lbrace\tilde{\mathbf{z}}_j^\prime\rbrace, \mathring{\mathbf{v}}^p)$ via Eq.(4) 4. Encode text embeddings $\mathbf{w}^p = \mathcal{T}(\mathbf{v}^p)$, $\mathbf{w}^s = \mathcal{T}(\mathbf{v}^s)$ 5. Train with loss functions $\mathcal{L}(\mathbf{w}^p, \mathbf{w}^s, \lbrace\mathbf{x_i}\rbrace, \lbrace\tilde{\mathbf{z}}_j\rbrace)$ as Eq.(1) and Eq.(3) The random OOD sample generator $\mathcal{G}$ can be distance-based $^{[1]}$, density-based $^{[2]}$, etc. Kindly note that the perturbation guidance itself does not involve gradient back-propagation, and we only optimize perceptual contexts $\mathbf{v}^p$ and spurious contexts $\mathbf{v}^s$. > 2. How is the perturbation implemented to help zero-shot classification? For zero-shot classification, we apply perturbation on pre-defined category descriptions (viewed as perceptual context $\mathbf{v}^p$) to simulate the spurious context $\hat{\mathbf{v}}^s$. Give an input image $x$, the classification process can be expressed as: 1. Perturb the perceptual context $\hat{\mathbf{v}}^s = \mathcal{P}(\mathbf{v}^p)$ 2. Encode text embeddings $\mathbf{w}^p = \mathcal{T}(\mathbf{v}^p)$, $\hat{\mathbf{w}}^s = \mathcal{T}(\hat{\mathbf{v}}^s)$ 3. Get initial image-text similarity $s = \langle \mathbf{w}^p, x \rangle$ 4. Compute regularization item $\gamma = \mathcal{R}(\mathbf{w}^p, \hat{\mathbf{w}}^s, x)$ via Eq.(5) 5. Compute regularized similarity $r = s \times \gamma$ 6. Determine the category $k = argmax_{k} \lbrace r_k \rbrace $ We will add more detailed pseudo-codes or flow charts in the revised paper. For reproducibility, the source code will be released upon acceptance. And we are also happy to answer any remaining or follow-up questions to clarify our method. [1] Leitian Tao. Non-Parametric Outlier Synthesis, ICLR 2023. [2] Xuefeng Du. VOS: Learning What You Don't Know by Virtual Outlier Synthesis, ICLR 2022. --- Best regards, Authors

Reviewer oU432023-08-16

Thanks for the response

I would like to thank the author for their rebuttal. My concerns are largely resolved. I would strongly recommend paraphrasing the usage of the term hierarchical, as I see multiple reviewers raising this question.

Authorsrebuttal2023-08-17

Thanks for the constructive feedback

We sincerely thank the reviewer again for evaluating our work and providing constructive feedback. We did neglect rigorous definitions of the term "hierarchical" in the initial manuscript. In the revised version, we will paraphrase the usage of "hierarchical" in the introduction/abstract section. --- Best regards, Authors

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC