Achieving Domain-Independent Certified Robustness via Knowledge Continuity

Summary

In this work, the authors propose a novel definition of "continuity": knowledge continuity, which measures the continuity of a model with respect to its hidden representations instead of the inputs. The knowledge continuity can be used to certify adversarial robustness across domain modality. Then, they provide theorems of robustness certification based on the new definition of continuity and demonstrate that in some cases the knowledge continuity implies Lipschitz continuity. Further, they prove that knowledge continuity doesn't affect the universal approximation property of neural networks, which means constraining a network to be knowledge-continuous may not hinder its expressiveness. Lastly, the authors provide some applications of knowledge continuity in improving robustness.

Strengths

* The novel definition of continuity is more general and can be used in discrete domains such as natural language. * The authors provide solid and interesting theories to support the value of knowledge continuity. * Some applications are provided to further prove their new definition's effectiveness.

Weaknesses

Some typos: * In Line 193, delete "with respect to a function $f$"? * In Line 218, $P_X[A]$ -> $P_{X\times Y}[A]$.

Questions

In this work, the authors decompose the network $f^k$ as $h^k\circ h^{k-1}\circ \dots \circ h^1\circ h^0$. However, some architectures such as ResNet and transformers, have residual structures. So, how can we decompose these networks as above?

Rating

7

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

As the authors said, this work provides some probability results that don't protect against OOD attacks. More deterministic results are needed in future studies.

Summary

This paper proposes a generalization of adversarial robustness and certified robustness for continuous, discrete, or non-metrizable input domains. The authors introduce _knowledge continuity_, a novel definition inspired by Lipschitz continuity which aims to certify the robustness of neural networks across input domains (such as continuous and discrete domains in vision and language, respectively). While existing approaches that aim to certify robustness (e.g., Lipschitz continuity) lie in the continuous domain with norm and distribution-dependent guarantees, the new proposed framework proposes certification guarantees that depend only on the loss function and the intermediate learned metric spaces of the neural network. The current definition of robustness considers the stability of a model's performance with respect to its perceived knowledge of input-output relationships. However, the authors argue that robustness is better achieved by focusing on the variability of a model's loss with respect to its hidden representations, rather than imposing arbitrary metrics on its inputs and outputs. To summarize the authors' contributions: - They introduce _knowledge continuity_, a new concept that frames robustness as variability of a model's loss with respect to its hidden representations. - They show theoretically that knowledge leads to certified robustness guarantees that generalize across modalities (continuous, discrete, and non-metrizable), and show that this new robustness definition does not come at the expense of inferential performance. - They present several practical applications of knowledge continuity such as using it to train more robust models and to identify problematic hidden layers.

Strengths

- The paper is clear and very well written. - The problem, i.e. defining a robustness framework for discrete or non-metrizable input domains, is very interesting and important as language models become more ubiquitous. - The new framework seems to generalize previous definitions of robustness, for example, robustness via Lipschitz continuity becomes a special case of knowledge continuity. - The authors show that their new definition of robustness - The authors propose experiments to demonstrate their approach on language tasks.

Weaknesses

- While the method seems to be useful for robustness in the context of discrete input domains such as language, the new framework also seems to inherit drawbacks of current certified methods, namely Lipschitz continuity and randomized smoothing: - Obtaining robustness from Lipschitz continuity requires having knowledge of the architecture of the network, and constraining the architecture to fit the definition; an example of this is that transformers are not _globally_ Lipschitz continuous, and thus certified robustness for transformer architectures via Lipschitz continuity is difficult. While knowledge continuity could in theoretically be used on any architecture where the hidden layers are metric spaces, I wonder if specific architectures might lead to vacius bounds. Similarly, quantized networks would not fall under the definition of having hidden layers that are metric spaces . - As randomized smoothing, the robustness derived from knowledge continuity is not deterministic. This could lead to a high computation of robustness accuracy and certified accuracy. - The implementation of knowledge continuity is not very clear, I understand that the authors have proposed a regularization scheme via Algorithm 1 for "Estimating knowledge continuity". Algorithm 1 estimates the the expected value that defines knowledge continuity, is this approximation accurate and how does the accuracy of the approximation affect the training speed? - While the authors have shown that there is theoretically no tradeoff between the robustness of knowledge continuity and natural accuracy, this does not mean that there is no tradeoff in practice. It would be interesting to see the tradeoff between between the strength of the attacks and the strength of the regularization, which seems to be missing from the paper. - The authors proposed a certified robustness approach, but no certification algorithm was provided. How to compute the certified accuracy with knowledge continuity. Given the non-deterministic approach of their framework, some statistical testing, as in randomized smoothing, may be necessary.

Questions

See Weaknesses.

Rating

7

Confidence

3

Soundness

4

Presentation

4

Contribution

3

Limitations

See Weaknesses.

Summary

This paper proposes a new concept inspired by Lipschitz continuity, aimed at certifying the robustness of neural networks across different input domains, such as continuous and discrete domains in vision and language. This certification guarantees based on the loss function and intermediate learned metric spaces, independent of domain modality, norms, and distribution. In this paper, the authors define knowledge continuity as the stability of a model's performance to its perceived knowledge of input-output relations. This approach focuses on the variability of a model's loss to its hidden representations rather than imposing arbitrary metrics on inputs and outputs.

Strengths

- The paper generalizes the Lipschitz continuity concept to continuous and discrete domains. - The paper provides definitions and proofs for knowledge continuity. - By providing a domain-independent approach to certified robustness.

Weaknesses

- The authors claim that their method generalizes across modalities (continuous, discrete, and non-metrizable). However, they have not conducted experiments in other domains. The experiments in Table 1 are limited to the discrete text domain. - The paper does not provide specific implementation details. Section 5 directly presents the experimental results in Table 1, but the metrics in Table 1 are not clearly defined. - From the paper "On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective," it is observed that model robustness improves with increased model capacity. It is necessary to demonstrate that the proposed method works on larger models like LLaMA3-70B/Qwen-1.5-70B to prove that the effectiveness of the method is not merely due to limited model capacity. - The proposed method does not consistently achieve better performance on ANLI. It remains competitive with previous methods rather than showing clear superiority. - The datasets used for comparison, IMDB and ANLI, focus more on word-level attacks, which are not particularly challenging for current large language models (LLMs). There is a lack of analysis on LLMs. - It is unclear how much benefit this method provides to bidirectional attention models like BERT and RoBERTa, which already have some error correction capabilities. - When dealing with out-of-distribution (OOD) data, the distributions themselves are different. It is uncertain whether this method will still be effective under such conditions.

Questions

- See "Weaknesses" - The equation in line 252 is not numbered and contains a typo.

Rating

5

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

N/A

Summary

This paper proposes "knowledge continuity", a metric inspired by the Lipschitz continuity that aims to certify the robustness of neural networks. Compared to existing methods, the proposed metric yields certification guarantees that only depend on loss functions and the intermediate learned metric spaces of neural networks, which are independent of domain modality, norms, and data distribution. The authors further show that the knowledge continuity is not conflicted with the expressiveness of a model class. Several practical applications are provided.

Strengths

- The metric seems to be novel, has an elegant form and seems to be able to provide good explanation of robustness. - The proposed metric is supported by a good amount of theoretical proofs, stating that improving the knowledge continuity does not be conflicted with the representation power of the neural networks. - practical results indicate that regularizing the metric can help the robustness.

Weaknesses

- A lot of details of the empirical results are deferred to the appendix which makes me hard to understand what are the experimental settings. The authors should consider reorganize the paper - right now it seems that the paper is actually compressed from a longer version (say journal) but not dedicated to the conference. For example, the regression test seems to be very interesting but there is no any actual result shown in the main text. - Some numbers are wrongly bolded in the table (or the meaning of bolded results is not clearly conveyed) since they are not the highest number.

Questions

I don't have questions.

Rating

7

Confidence

2

Soundness

3

Presentation

2

Contribution

3

Limitations

Yes, the authors have discussed the limitation in the main text.

Thanks for your reply

The authors' reply has answered my questions. And I keep the score unchanged.