```
Comments: It is true that the proposed transformer which is based on FA is more efficient than Equiformer which is based on higher-order tensors, which is an advantage. However, to my knowledge, the idea of such architectural design is not so novel.
```
**Ans**: We believe the key contribution and novelty of our architecture lie in demonstrating the potential of designing and applying equivariant modules based on FA, which, to the best of our knowledge, has not been explored previously. Moreover, the primary motivation behind our architectural design is to address the inherent issues of the prior Transformers, as mentioned in Lines 35-38, which is non-trivial and can effectively enhance performance.
```
Comments: And some important baselines are missing, e.g., [1].
```
**Ans**: Thank you for bringing this to our attention! We have applied GATr’s official implementation and evaluated its performance on the contact map prediction task, as shown below:
| | | EGNN | GATr | FAFormer |
| --- | --- | --- | --- | --- |
| Protein-RNA | F1 | 0.1093 | 0.1107 | 0.1284 |
| | PRAUC | 0.0964 | 0.0976 | 0.1113 |
| Protein-DNA | F1 | 0.1208 | 0.1077 | 0.1457 |
| | PRAUC | 0.1139 | 0.1075 | 0.1279 |
The input features for GATr are consistent with those used in our method and other baselines, including coordinates and pretrained residue/nucleotide embeddings. Following GATr’s setup, the input coordinates are first mapped to tri-vectors. However, GATr’s original implementation is designed for general point clouds and uses full attention among points, which is memory-inefficient for large biomolecules. To address this, we limited the attention to K-nearest neighbors. The hyperparameters were tuned using the same process outlined in Appendix A and are shown below:
- Hidden dim: 64
- Hidden dim for multivector: 32
- Number of neighbors: 30
- Number of attention blocks: 5
- Dropout ratio: 0.2
```
Comments: I wonder the performance of the model on SE(3)-equivariant property prediction tasks. To be more precise, contact map prediction, binding site prediction, fold prediction and reaction prediction are SE(3)-invariant property prediction tasks, while protein-protein docking or protein-ligand docking tasks that I mentioned in the previous review can be formulated as SE(3)-equivariant prediction tasks with well-established datasets and benchmarks. I highly recommend the authors to test the proposed architecture on these tasks.
```
**Ans**: Existing docking benchmarks [1,3] primarily compare different docking models, such as diffusion model-based approaches [4] and alignment-based models [2]. FAFormer, as a geometric backbone, is not directly comparable to these docking methods but is **compatible** with them by extending or replacing the geometric encoder they use.
To demonstrate this compatibility, we present the results of integrating FAFormer with Equidock [2] on the protein-protein docking task using our protein complex dataset:
| | complex RMSD | interface RMSD |
| --- | --- | --- |
| Equidock | 20.266 | 22.769 |
| Equidock w/ FAFormer | 18.572 | 18.089 |
In this table, "complex RMSD" represents the mean distance between each pair of residues, while "interface RMSD" focuses specifically on the binding interface. These preliminary results demonstrate that incorporating FAFormer can effectively enhance the performance of existing docking methods.
We believe that extending FAFormer to different fields, such as structure prediction and docking, is a promising research direction that we plan to explore further. A discussion will be added to our updated manuscript.
[1] Deep Learning for Protein-Ligand Docking: Are We There Yet?
[2] Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking
[3] Accurate structure prediction of biomolecular interactions with AlphaFold 3
[4] DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking