Reply to Reviewer fVht
Thanks for your replies. Please find our responses below.
> [Reply to Q1] Kindly note that, to the best of our knowledge, we are the first effort to connect causal reasoning with ACL. We construct a novel causal graph of ACL and provide the theorem to justify the rationality of our constructed causal graph of ACL.
> [Reply to Q2] Details of the t-test.
In Table 1 (a.k.a. ***Table A***), for **baselines** (i.e., ACL and DynACL) and **our proposed method** (i.e., ACL with IR and DynACL with IR), we used a single pre-trained checkpoint and repeated the finetuning procedure three times. Therefore, for each baseline and each dataset, we have **three** results of robust test accuracy and **three** results of standard test accuracy. Note that, for baselines, we used the public pre-trained checkpoints downloaded from their official GitHub.
To conduct t-tests, we used the following code:
```
from scipy import stats
a = the list of results of the baseline
b = the list of results of our proposed method
if stats.levene(a, b).pvalue > 0.05: # check whether the variance is the same.
print(stats.ttest_ind(a,b,equal_var=True).pvalue) # check whether there is significant difference between two distributions.
else:
print(stats.ttest_ind(a,b,equal_var=False).pvalue)
```
On each dataset, we conducted a t-test between the three results of robust/standard test accuracy achieved by the baseline and the three results of robust/standard test accuracy achieved by our proposed method. Then, we reported the corresponding p-value in ***Table A***. We can observe that the p-value is far smaller than 0.05, which means that the t-test rejects the null hypothesis and our results are significantly better than the baseline.
> [Reply to Q3] Perhaps there is a misunderstanding. Kindly note that SIR is the method of [1]. **AIR is *our* proposed regularization**. IR means AIR+SIR.
As you mentioned, our proposed AIR is apparently more effective than SIR since AIR can enhance both standard and robust test accuracy more significantly compared to SIR.
We empirically found that IR slightly improves performance compared to AIR. Therefore, we choose to use SIR and our proposed AIR together.
> [Reply to Q4] IR regulates the representations to be invariant of style factors, thus enhancing the robustness in downstream tasks.
We can treat adversarial attacks and common corruptions as style factors which are functions to manipulate the original data without changing its semantics. Adversarial attacks add imperceptible input perturbations to the original data while maintaining the original semantics. Common corruptions [2] used 15 types of algorithms from noise, blur, weather, and digital categories to generate input perturbations without modifying the semantics of the original data.
IR aims to regulate the representations to be invariant of style factors. Therefore, IR helps to make the representations robust against style factors that do not change the semantics, thus improving robustness against adversarial robustness and common corruptions.
*References*\
[1] Representation learning via invariant causal mechanisms. Mitrovic et al., ICLR 2021.\
[2] Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. Hendrycks et al., ICLR 2019.