Summary
This manuscript proposes several methods for identifying the primary structure, or nucleic acid sequence, of an RNA molecule given its tertiary structure, or three-dimensional structure. RNA structure prediction, for reasons that will be discussed, is studied much less than protein structure prediction. This discrepancy presents two major issues: Firstly, there are not enough high-quality RNA structures to build models from, and secondly, there are not as many methods developed to predict RNA structure. Therefore, the authors initially compile and refine a dataset, carefully dividing it into training and testing sets, and then develop new algorithms to address this problem. Since there are no existing methods to predict the RNA sequence from its structure, the authors not only propose a novel method — inspired from Ingraham _et al._ (2019) — but also introduce five baseline methods for comparison. Finally, they test their models on two significant, completely independent datasets: Rfam and RNA-Puzzles.
Strengths
The authors developing this method — the first of its kind — have done an impressive job on multiple fronts:
- Although the proposed method is an extension of the "protein structure to sequence" method proposed in Ingraham et al. (2019), RNA molecules are different from protein molecules, featuring more dihedral angles and a different local geometry. This necessitates the development of new encoding schemes.
- The attention and care the authors have devoted to compiling a clean RNA structure dataset are noteworthy. The training and testing splits are carried out in a structurally-informed manner, which is crucial in structural biology. Additionally, they have maintained similar length distributions.
- Given the limited availability of RNA structures, the authors employ a hierarchical representation learning scheme to address this challenge. This approach groups similar clusters together, much like protein fold families. They also utilize data augmentation by making slight alterations to RNA structures and ensuring their proximity in the latent space.
- The ablation study is informative and serves to justify the modeling decisions made by the authors.
- Impressively, the authors have also developed five baseline models.
Weaknesses
Proteins are arguably not structurally simpler than RNAs, and a reader might get this impression from reading this manuscript. Most RNAs lack a defined structure, and the majority of RNAs do not function primarily through their structures. Even a large number of small non-coding RNAs function through base pairing, such as miRNAs. I believe the authors should demonstrate a tangible application for their proposed method. Is it intended for designing RNA aptamers? If so, how and why would it be superior to SELEX methods?
Minor Issues:
- Figure 4 could be made more clear; the green arrows appear too small. Additionally, clusters and samples within clusters should be better aligned.
- The last sentence of the second paragraph on page 2 is incomplete.
Questions
I decided to move some of the perceived weaknesses to the questions:
- The base atoms are masked; however, purines (R or A/G) and pyrimidines (Y or C/T) differ significantly in size, and the model could have simply learned to predict the base using the spacing between the backbone atoms of neighboring nucleotides. This is not as significant of an issue in proteins, as there are twenty different side chains that do not canonically pair with each other. One way to test this is to evaluate performance by converting all A/G nucleotides to R and all C/T nucleotides to Y, then report recovery scores based on R and Y only. If this score is substantially higher than the original one, then the model may simply be learning whether bases are purines or pyrimidines.
- When constructing the neighborhood graph, wouldn't it be more intuitive to use a ball around each nucleotide? Could enforcing a fixed K result in some edges that are too distant in space?
- Can't this method be iterative? That is, determine the primary sequence from the structure, then compute secondary and tertiary structures from the inferred primary sequence, identify discrepancies, and predict a new primary sequence, repeating this process until a certain condition is met.
- Related to the previous question, is the "recovery score" the best metric? Is the goal to predict THE primary sequence or A primary sequence that gives rise to a very similar tertiary structure? Recovery scores, in general, tend to be low, and reporting predicted RMSD (e.g., using RhoFold) might be helpful in this regard.
- RNA structure is not entirely rigid, and some techniques like NMR can capture an ensemble of structures. Would all such structures in the bundle get the same sequence?
- Approximately 90% of RNA sequences are short (under 100 nt), and a Bayesian Optimization method might work well for this problem. It involves obtaining the tertiary structure, computing the secondary structure, starting with a random sequence, computing its secondary structure, and using the model to propose a new sequence until a close enough solution is found.
- Can Cohen's Kappa be used instead of or in addition to Macro F1?
Rating
6: marginally above the acceptance threshold
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.