Summary
Based on the belief that learning with fragments can help capture structural information at multiple resolutions, this paper proposes a fragment-based strategy for pretraining and fine-tuning.
First, the paper extracts fragments by an existing heuristic algorithm called Principle Subgraph Mining algorithm to obtain fragments from a large molecular dataset (i.e., ChEMBL database). Then, the paper uses two separate GNNs (one for molecules and one for fragments) to learn the node embeddings. In particular, the node embeddings obtained by molecular-based GNN are pooled into fragment embeddings.
The model is trained with respect to three tasks: a contrastive task between fragment embeddings obtained by molecular-based GNN and fragment-based GNN, a fragment existence prediction task, and a fragment graph structure prediction task.
The experiments are done on 8 binary graph classification tasks from MoleculeNet and 2 graph prediction tasks on large peptide molecules from the Long-range Graph Benchmark.
Strengths
+ The proposed method is easy to follow and conduct.
+ The results on Long-range Graph Benchmark are particularly good.
+ Figure 1 clearly shows the method.
+ Codes are provided. Appendix further provides more details.
Weaknesses
The idea of using molecular fragment can be interesting. The proposed method shows some effectiveness, although how it obtains can be less illuminating.
Many design choices need to be further described. Please reply to my questions below.
In addition, some writing issues exist. Sentences cannot start with "[reference]".
Questions
1. The fragment extraction strategy can be described and compared with more details. The authors choose Principle Subgraph Mining algorithm to extract fragments. Do you try others? Any empirical evidence supports this choice? Like BRICKS or RECAP?
On line 53, the authors comment MGSSL as "their multi-step fragmentation may overly decomposes molecules, losing the ability to represent high-order structural patterns". Can you expand it? Why?
On line 122, the authors wrote that "however, existing methods use suboptimal fragmentation or fragment embeddings". Likewise, can you provide more discussion about it? Why their methods obtain suboptimal fragments while yours can? Any theoretical proof?
2. The method assigns different fragment embeddings for two identical fragments appearing int the same molecule. Have you trying forcing them to be the same? I get that the local neighborhood of these two identical fragments can be different. But will it also help capture generic information if fragment is uniquely represented?
3. On lie 159, the authors wrote that "An edge exists between two fragment F nodes of GF is there exist at least a bond interconnecting atoms from the fragments." Will these cause too many edges between fragments? I suspect that most fragments are connected in this case. Therefore, the topology of fragment graph is lost.
4. How to obtain ground-truth structural backbones?
5. More experimental details of fragment extraction algorithm are needed. Any hyperparameters? On line 308, the authors wrote that "we prepare two additional vocabularies with size 1600 and size 3200". How do you prepare that?
6. Why results on long-range tasks can be much improved than existing works? Why different tasks use different baselines?
7. Figure 4(d) is hard to follow. How do you assign fragment ID? Why you need fragment ID as x axis?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
No potential negative societal impact of their work as far as I know.