Protein-Nucleic Acid Complex Modeling with Frame Averaging Transformer

Nucleic acid-based drugs like aptamers have recently demonstrated great therapeutic potential. However, experimental platforms for aptamer screening are costly, and the scarcity of labeled data presents a challenge for supervised methods to learn protein-aptamer binding. To this end, we develop an unsupervised learning approach based on the predicted pairwise contact map between a protein and a nucleic acid and demonstrate its effectiveness in protein-aptamer binding prediction. Our model is based on FAFormer, a novel equivariant transformer architecture that seamlessly integrates frame averaging (FA) within each transformer block. This integration allows our model to infuse geometric information into node features while preserving the spatial semantics of coordinates, leading to greater expressive power than standard FA models. Our results show that FAFormer outperforms existing equivariant models in contact map prediction across three protein complex datasets, with over 10% relative improvement. Moreover, we curate five real-world protein-aptamer interaction datasets and show that the contact map predicted by FAFormer serves as a strong binding indicator for aptamer screening.

Paper

References (92)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer npWJ7/10 · confidence 3/52024-07-09

Summary

The authors propose am method for predicting contacts between proteins and aptamers based on frame averaging transformers. They showcase their methods on contact prediction and unsupervised aptamer screening, showing improvements over some baselines. The authors also compare to RoseTTAFoldNA, which seems to outperform their method.

Strengths

- Frame averaging is a very interesting idea (published in another venue before) and applying it to this specific problem seems to be very promising - The presentation is very clear and the paper is well written - The problem addressed (aptamer screening) is very important and practically relevant - The authors compare to a large number of different architectures (see Table 4)

Weaknesses

Comparing to a pre-trained structure prediction method is clearly something to be explored well, and the authors do that in section 4.4 (with additional results with AF3 in the appendix). At the same time, this part seems to me not very well developed in the paper. I am especially confused by Figure 3, which the authors comment with "[..] where FAFormer can achieve comparable performance to RoseTTAFoldNA using unbounded structures." Maybe I am not understanding something but the right hand side of the plot seems to clearly indicate that at least for RNAs, RoseTTAFoldNA strongly outperforms the authors' method.

Questions

- Could the authors expand section 4.4 and discuss the combination of Figure 3 and Table 5? - Do the times in Table 6 include MSA search for Rosetta? If the protein is unchanged, could you make it more efficient by doing the MSA search only once? - Could the authors expand a bit on possible future research (e.g. applying the same method to other screening tasks)?

Rating

7

Confidence

3

Soundness

4

Presentation

4

Contribution

3

Limitations

Authors discuss some scientific limitations. I do not foresee any negative societal impact.

Reviewer qqru4/10 · confidence 3/52024-07-10

Summary

This paper mainly focuses on protein-nucleic acid contact prediction and unsupervised aptamer virtual screening. The latter is based on an unsupervised learning approach by predicting the contact maps. An equivariant architecture that integrates frame averaging and transformer blocks is proposed for this task. Experiments on the two tasks demonstrate the superiority of the proposed architecture compared with other geometric deep learning models.

Strengths

- The proposed architecture is effective on the contact map prediction tasks. The tasks are novel and not well-studied in the community of machine learning. - The unsupervised learning approach that predicts contact maps is effective on aptamer screening tasks. This perspective is novel. - The authors have provided codes and reproducibility can be ensured.

Weaknesses

- The novelty of the proposed architecture is limited. The proposed architecture simply combines frame averaging and transformer blocks. The idea that builds the attention module based on SE(3)-invariant features is proposed in Equiformer. The dataset and benchmark track may be more suitable for this paper. - The expressiveness of the proposed architecture is demonstrated in the contact prediction tasks. More experiments on more general tasks are needed to further show the effectiveness of the proposed architecture, such as protein-protein docking and others.

Questions

See the above weaknesses.

Rating

4

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors have pointed out the limitations due to only modeling certain atoms in the related tasks.

Reviewer DGm47/10 · confidence 3/52024-07-11

Summary

The authors propose a novel equivariant model, FAFormer, which leverages the frame averaging operation as an integral geometric component within the Transformer. The authors prove the invariance and equivariance of the architecture. They further conduct experiments showing that FAFormer performs well in contact map prediction and could serve as a strong binding indicator for aptamer screening.

Strengths

- FAFormer is novel in incorporating frame averaging within each layer of the Transformer. The FAlinear module provides a novel method for creating invariant models. - Empirical results demonstrate the superior performance of FAFormer on contact map prediction and aptamer screening tasks. FAFormer excels in screening aptamers and outperforms AlphaFold3. - The authors provide a detailed proof of the invariance and equivariance property.

Weaknesses

- The F1 score for contact map prediction tasks is about 0.1, which is very low. I question whether this score can be used to judge which model is better. The model potentially performs worse due to separate modeling of nucleic acid and protein, the linear prediction head, and the difficulty of the pairwise prediction task. I suggest involving other node-level and graph-level prediction tasks for comparing FAFormer with other equivariant neural networks. - The main idea of the model is confusing. If we already know the coordinates of the protein and nucleic acid, why do we need further encoding and prediction for contact prediction, instead of directly counting the contacts?

Questions

- The module structures with formulas are a bit difficult to follow. Also, could you provide more intuition on the design of these modules? - Since we are constructing a KNN graph and always leveraging the coordinate X and edge E information, why should the model be called a “former” instead of a GNN, which could be misleading? - As far as I know, RosettaFoldNA is used mainly for protein and NA structure prediction. However, FAFormer uses coordinate information as input. Is that a fair comparison? - I am curious about the efficiency of FAFormer compared with other equivariant NNs. Could you provide some analyses? - During experiments, do other methods include the same featurization? And are you training all models from scratch?

Rating

7

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

The work provides an innovative framework, FAFormer, and performs well on contact map prediction and aptamer screening tasks. However, the experimental setting is not convincing enough and requires refinement.

Reviewer npWJ2024-08-11

I thank the authors for addressing my concerns. My score (accept) remains the same.

Authorsrebuttal2024-08-12

Thank you again for providing valuable comments on our paper! All the suggestions will be added to our updated manuscript.

Authorsrebuttal2024-08-12

Thank you again for your hard work in reviewing our paper! We hope you've had a chance to review our responses to your comments. Please let us know if you have any further questions or concerns. We greatly appreciate your feedback and are committed to clarifying the model innovation and the evaluation of the additional benchmarks.

Reviewer qqru2024-08-12

Thanks for your response! It is true that the proposed transformer which is based on FA is more efficient than Equiformer which is based on higher-order tensors, which is an advantage. However, to my knowledge, the idea of such architectural design is not so novel. And some important baselines are missing, e.g., [1]. I wonder the performance of the model on SE(3)-equivariant property prediction tasks. To be more precise, contact map prediction, binding site prediction, fold prediction and reaction prediction are SE(3)-invariant property prediction tasks, while protein-protein docking or protein-ligand docking tasks that I mentioned in the previous review can be formulated as SE(3)-equivariant prediction tasks with well-established datasets and benchmarks. I highly recommend the authors to test the proposed architecture on these tasks. My score currently remains the same. References: [1] Brehmer, Johann, et al. "Geometric algebra transformer." NeurIPS 2023

Authorsrebuttal2024-08-13

``` Comments: It is true that the proposed transformer which is based on FA is more efficient than Equiformer which is based on higher-order tensors, which is an advantage. However, to my knowledge, the idea of such architectural design is not so novel. ``` **Ans**: We believe the key contribution and novelty of our architecture lie in demonstrating the potential of designing and applying equivariant modules based on FA, which, to the best of our knowledge, has not been explored previously. Moreover, the primary motivation behind our architectural design is to address the inherent issues of the prior Transformers, as mentioned in Lines 35-38, which is non-trivial and can effectively enhance performance. ``` Comments: And some important baselines are missing, e.g., [1]. ``` **Ans**: Thank you for bringing this to our attention! We have applied GATr’s official implementation and evaluated its performance on the contact map prediction task, as shown below: | | | EGNN | GATr | FAFormer | | --- | --- | --- | --- | --- | | Protein-RNA | F1 | 0.1093 | 0.1107 | 0.1284 | | | PRAUC | 0.0964 | 0.0976 | 0.1113 | | Protein-DNA | F1 | 0.1208 | 0.1077 | 0.1457 | | | PRAUC | 0.1139 | 0.1075 | 0.1279 | The input features for GATr are consistent with those used in our method and other baselines, including coordinates and pretrained residue/nucleotide embeddings. Following GATr’s setup, the input coordinates are first mapped to tri-vectors. However, GATr’s original implementation is designed for general point clouds and uses full attention among points, which is memory-inefficient for large biomolecules. To address this, we limited the attention to K-nearest neighbors. The hyperparameters were tuned using the same process outlined in Appendix A and are shown below: - Hidden dim: 64 - Hidden dim for multivector: 32 - Number of neighbors: 30 - Number of attention blocks: 5 - Dropout ratio: 0.2 ``` Comments: I wonder the performance of the model on SE(3)-equivariant property prediction tasks. To be more precise, contact map prediction, binding site prediction, fold prediction and reaction prediction are SE(3)-invariant property prediction tasks, while protein-protein docking or protein-ligand docking tasks that I mentioned in the previous review can be formulated as SE(3)-equivariant prediction tasks with well-established datasets and benchmarks. I highly recommend the authors to test the proposed architecture on these tasks. ``` **Ans**: Existing docking benchmarks [1,3] primarily compare different docking models, such as diffusion model-based approaches [4] and alignment-based models [2]. FAFormer, as a geometric backbone, is not directly comparable to these docking methods but is **compatible** with them by extending or replacing the geometric encoder they use. To demonstrate this compatibility, we present the results of integrating FAFormer with Equidock [2] on the protein-protein docking task using our protein complex dataset: | | complex RMSD | interface RMSD | | --- | --- | --- | | Equidock | 20.266 | 22.769 | | Equidock w/ FAFormer | 18.572 | 18.089 | In this table, "complex RMSD" represents the mean distance between each pair of residues, while "interface RMSD" focuses specifically on the binding interface. These preliminary results demonstrate that incorporating FAFormer can effectively enhance the performance of existing docking methods. We believe that extending FAFormer to different fields, such as structure prediction and docking, is a promising research direction that we plan to explore further. A discussion will be added to our updated manuscript. [1] Deep Learning for Protein-Ligand Docking: Are We There Yet? [2] Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Docking [3] Accurate structure prediction of biomolecular interactions with AlphaFold 3 [4] DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking

Authorsrebuttal2024-08-13

Thank you once again for your valuable comments on our paper! As the deadline is approaching, we may not be able to respond after that. Please let us know if you have any further questions or concerns. We are committed to addressing your feedback.

Reviewer qqru2024-08-14

Thanks for providing the experimental results! I have two related questions: - Why not use the standard protein complex dataset? In the dataset you proposed in the paper, there are only about 4k protein-protein complexes, while in the widely used protein-protein complex dataset, i.e., the Database of Interacting Protein Structures (DIPS), there are about 40k samples. The experiments on such a small dataset may not be very convincing. - The results of complex RMSD and interface RMSD are too high. In this case, the numerical results (~18) may indicate invalid docking results. Please refer to the paper of EquiDock for more details. Their results are much lower than the reported in the above response.

Reviewer DGm42024-08-12

Thank you for the responses. I believe all of my concerns have been addressed, and I look forward to seeing additional results/discussion in the updated manuscript. I have raised my score from 5 to 7.

Authorsrebuttal2024-08-13

We greatly appreciate your valuable comments on our paper! All the suggestions will be added to our updated manuscript.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC