Dear Reviewer,
We appreciate the time you've taken to review our work. We kindly direct your attention to the points we made in our reply.
- Our contribution is to the pre-training framework, not the model architecture itself. ET-OREO has significantly improved the performance of ET, and as we have shown in the tables we submitted in the reply, outperforms NequIP by as much as 40%, with significantly reduced training time and inference time and greater versatility to finetune on different molecules. This should be enough for one to say that the pre-training framework of ET-OREO is effective.
- Again, our contribution is the pre-trained framework that hinges on forces. This includes the joint training of forces on off-equilibrium data and the de-noising objective on equilibrium data as an approximation. We will make changes to our claims regarding training with forces, see the end of this reply.
- It is also well-known that pre-trained models work better in larger datasets with deeper models and more parameters. Of course, NequIP is shown in the existing paper to be a strong baseline when for a small model data are limited, but neither is the case in the context of pre-training. We re-iterate that our main contribution is the pre-training framework, which in principle is independent of the model architecture of choice, and the choice of ET is based on the consideration of scalability, inference speed, and the size of model parameter space. The inference time of NequIP makes it more difficult for a pre-training task.
- We are still training a NequIP model with our pre-training framework. It takes a significantly longer time, and we will share the results as soon as possible.
In addition to the results we reported previously, upon further tuning ET-OREO to 2000 epochs, we recorded even better performance metrics on ET-OREO than previously shared:
- Aspirin: 1.0 meV/A (52% improvement over NequIP's 2.1 meV/A)
- Ethanol: 1.0 meV/A (28% improvement over NequIP's 1.4 meV/A)
- Naphthalene: 0.9 meV/A (18% improvement over NequIP's 1.1 meV/A)
- Salicylic Acid: 0.9 meV/A (10% improvement over NequIP's 1.0 meV/A)
Thank you for your constructive feedback, and we are looking forward to your reply.
----------------------
Here's the revised version of the third paragraph in the Introduction where the reviewer expressed several concerns about the claim of the paper and language. We have clarified the concept of forces being generalizable and removed the use of language that suggests that we were the first to leverage forces.
In this paper, we incorporate both off-equilibrium and equilibrium conformations into a unified representation learner.
We propose a new pre-training model, "ET-OREO'" for 3D molecular conformations.
This model integrates both equilibrium and off-equilibrium data from multiple large-scale datasets.
Its training objective hinges on \emph{atomic forces}, defined as the negative gradient of a molecule's potential energy with respect to atomic coordinates.
Atomic forces exhibit several notable properties: (1) they are \emph{physically well-defined} observable, i.e., the force acting on an atom is determined solely and uniquely from its local environment, defined as the real-space distribution of its neighboring atoms; (2) they are generalizable across various molecules in the sense that atoms from different molecules that have the same local environment should experience the same atomic forces; and (3) they can unify equilibrium and off-equilibrium data, as equilibrium data can be conceptualized as local minima in the latent (configuration) space with zero forces, while off-equilibrium data aid the model in more accurately characterizing the high-energy chemical space beyond equilibrium. Among these points, (2), which is the direct consequence of (1), makes atomic forces fundamentally different from potential energy which is defined for the whole system under consideration (molecules) and can only be determined up to an additional constant. Therefore, a predictive model of atomic forces is transferable, i.e., in principle, it can be used directly for molecules of any size (number of atoms). This advantage makes the "learning atomic forces"' fundamentally different from the traditional approach in which a fictitious concept of ``atomic energy'' must be defined, predicted, and combined to obtain the total potential energy of the whole system.