Thanks for your response.
We greatly appreciate your reply and want to address your concern as much as possible.
**Q1**: The VRA optimal operation is more like a simple variant of existing feature clipping methods, such as ReAct, BAST, ASH. ReAct clips high activations, BAST clips both low activations and high activations and ASH-B clips low activations. Although the author emphasizes the difference from BAST, in fact VRA only add three hyper-parameters, while BAST utilized mean and variance to get the min and max thresholds.
**A1**: Thank you very much for your valuable comments. In addition to our responses to Reviewer J9PQ (see A1) and Reviewer JjUp (see A2), we also try to restate the difference between VRA-based methods with existing approaches. In this paper, we extend ReAct and propose new activation functions in OOD detection. VRA-based approaches are not simple variants of existing approaches but have strong motivations and theoretical guarantees. Compared with ReAct, BAST, and ASH which merely truncate low and high activations, we further prove the necessity of amplifying intermediate activations. Such a simple modification can achieve state-of-the-art OOD detection performance, which fully validates its effectiveness. As Reviewer JjUp comments, our VRA-based methods "allow more possibilities to perform OOD detection".
**Q2**: There are still some problems in the interpretation of formula 3. ReAct clips the high activations and proves that the average activation reduction of OOD is larger than the reduction of ID, which is reasonable. Eq.3 is lack of insight, the author may provide more explanations about why the objective function (Eq.3 ) benefits OOD performance.
**A2**: Thanks for your comments and we apologize for our unclear description. First, let's review the theoretical analysis in ReAct (see Section 5 of ReAct ). It proves that the rectification operation affects OOD activations more severely compared to ID activations and results in a large $ E_{out} ( z_i - \bar{z_i} ) - E_{in} ( z_i - \bar{z_i} ) $ (see Remark 1 of ReAct). The increased separation between OOD and ID activations can transfer to the output space as well (see Remark 2 of ReAct), thus enlarging the gap between OOD and ID score. Rather than just increasing separation like ReAct, this paper attempts to maximize the separation. Based on this motivation, we propose the objective function in Eq. 3. Experimental results on benchmark datasets also demonstrate the effectiveness of our method.
**Q3**: The optimal operation is based on feature histograms of ID set and OOD set. For different models and OOD datasets, the estimated optimal operation functions might be different, which makes it difficult to obtain the optimal function. For example, the activation functions on iNaturalist, SUN and CIFAR-100 are quite different.
**A3**: Thanks for your valuable comments. In Figure 2 (see attached PDF in Global Response) and Figure 1 (see the main text), we provide visualization results on multiple datasets and obverse the same phenomenon. From this phenomenon, we verify the necessity of suppressing abnormally low and high activations and amplifying intermediate activations. Although different models and OOD datasets might have different optimal operation functions, we can adjust the hyper-parameters in activation functions to approximate the optimal operation function.
**Q4**: Besides, it seems that the optimal function is estimated based on the test ID/OOD set, rather than a validation set.
**A4**: In fact, we use Gaussian noise images as the validation set for hyper-parameter tuning (see Implementation Details in the main text). If we choose the best hyper-parameter according to the test ID/OOD set, we can achieve even higher OOD detection performance. For example, we set $\eta_{\alpha} = 0.6$, $\eta_{\beta} = 0.95$ and report "17.74/96.47(FPR95/AUROC)'' in Table 1 of our main text. If we choose the best hyper-parameter based on the test ID/OOD set, we can get "16.78/96.63 (FPR95/AUROC)''.