Neural Pfaffians: Solving Many Many-Electron Schrödinger Equations

Summary

In this paper, the authors propose a novel Pfafifian-based anti-symmetrization method to represent the generalized wavefunction in quantum chemistry. Unlike the traditional Slater determinant-based anti-symmetrization, the Pfaffian-based method offers greater flexibility in selecting the number of orbitals. This flexibility is crucial for generating a universal wavefunction representation for molecular systems with varying number of electrons. The authors also introduce a new pretraining scheme that addresses the rotational symmetry in Hartree-Fock (HF) solutions, thereby improving the quality of the initial guess in their method. They validate the effectiveness of their approach through experiments on atomic systems and the TinyMol dataset.

Strengths

1. To the review's knowledge, this is the first study that uses Pfaffian as an anti-symmetrization method in the area of solving the Schrödinger equation with neural networks. 2. The proposed pretraining scheme effectively avoids orbital disorder in HF solutions, ensuring that the initial guess for the wavefunction is more accurate and stable. This scheme can be integrated with the other methods in solving the Schrödinger equation, facilitating further development and refinement in this area. 3. The performance of the proposed method is not affected by the uncorrelated data, indicating its potential to incorporate more training data to improve the transferability of the proposed method.

Weaknesses

1. The primary techniques in this paper are the Pfaffian anti-symmetrization method and the associated pretraining scheme. However, in the TinyMol dataset evaluation, the equivariant part of the proposed method differs from that of the baseline, making it difficult to attribute performance difference solely to the new anti-symmetrization method. Additionally, the performance of the proposed method is not consistently superior to the baseline. The reviewer recommends an ablation study to clearly demonstrate the effectiveness of the proposed method. 2. To handle odd numbers of electrons, the authors introduce a learnable padding vector to prevent Pfaffian collapse. While this approach appears effective in in-distribution experiments (e.g., affinity and ionization potential of atomic systems), the reviewer is concerned about its efficacy in out-of-distribution scenarios, particularly when generalizing to unseen systems with different electron numbers.

Questions

There are other ways to construct a skew-symmetric matrix, such as $\Phi_1^{\top}\Phi_2-\Phi_2^{\top}\Phi_1$, where $\Phi_1,\Phi_2\in\mathbb{R}^{N_{0}\times N}$ are the output of different equivariant networks. The reviewer is curious about why the authors choose $\Phi^{\top}A\Phi$ in Neural Pfaffians.

Rating

6

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

See weakness part

Summary

This paper proposes a new ansatz (Neural Phaffian) for parameterizing wave functions. The new ansatz improves the expressive power by making it possible to increase the number of orbitals. It is also beneficial to tasks like generalization between different systems. The effectiveness is demonstrated with plenty of experiments.

Strengths

- The paper targets important tasks in physical sciences. It is laudable that the authors not only address ground-state energy calculation but also consider ionization energy and generalization among different systems, which are of more practical significance. - Handling anti-symmetry with Pfaffian is a very novel and clever idea! How to enforce anti-symmetry is one of the most important problems in this task, and for decades we have not been so far from original Slater determinants. This work provides rich insights and opens up great opportunities for future research. - The authors present comprehensive empirical studies to demonstrate the advantages of their model.

Weaknesses

- As mentioned in L250-258, Neural Pfaffian is slower than a comparably-sized wave function with Slater determinants. It would be better if the authors could add training-convergence plots with time instead of iterations when comparing Neural Pfaffian with baseline methods. - The metric of summing over all energies in a dataset, as in Fig 7, is quite weird. I do not think this metric makes much sense. Additionally, it is possible that some systems with large absolute energy values dominate the curve. It is not convincing enough to showcase the advantage in **most** systems in the dataset. - There are several typos. I highly recommend the authors read the paper thoroughly again to fix all the typos. Just to list some of them: 1. L23: It should be $ \langle \Psi | \hat{H} | \Psi \rangle$. 2. L120: ‘inferring’.

Questions

1. Is it obvious that Neural Pfaffian is better than the sum of parameterized orbital determinants $\sum_k |(\phi_{i,k}(r_j;r_{-j}))_{i,j}|$ that has a similar number of parameters? By ‘better’, I mean both expressive power (i.e. the least energy that the model can attain with sufficiently long optimization time) and convergence (i.e. the energy achieved within a fixed time period/number of iterations). 2. Regarding the ablation study on envelopes, it turns out that the full envelope(green) is less costly per iteration than the red and yellow curves. This is weird because the model with efficient envelope contains fewer computations. Furthermore, the red line reaches lower energy than the green line. This is also weird because the model/wave function with full envelope is richer in expressiveness and thus has the potential to attain a lower energy. Please correct me if I misunderstood this part. 3. Could the authors give a possible explanation on why Neural Pfaffian’s result gets significantly worse in the 2-hydrogen case? Intuitively this setting is easier than the case with a larger number of hydrogens. 4. Regarding L157, when the paper addresses systems with odd numbers of electrons, the approach of concatenating an additional learnable vector appears unnatural. The structure of the wave functions would change significantly when a system gains or loses an electron. Overall, I enjoy reading this work and vote for a strong accept.

Rating

8

Confidence

3

Soundness

4

Presentation

3

Contribution

4

Limitations

Yes.

Summary

In this paper, the authors propose using Pfaffians instead of Slater determinants to learn a generalized neural wave functions so that the permutation antisymmetry enforcement can be better addressed. Empirical study shows one single proposed model can generalize to various systems (second-row elements) with chemical accuracy. For the nitrogen potential surface prediction, the proposed model outperforms previous work Globe. The proposed model outperforms CCSD(T) and previous work TAO on the TinyMol dataset which contains hundreds of samples.

Strengths

- By replacing Slater determinants, the proposed method avoids discrete and manual orbital selections, and hence is overparameterized, fully learnable, and applicable to any molecular systems. - Techniques such as memory-efficient envelopes, pretraining by Hartree-Fock, and generalization, are investigated to improve the efficiency and application of the proposed method.

Weaknesses

I am not capable of discovering any weaknesses beyond the ones listed in the Limitation section, or the limitation of the entire neural wave function domain.

Questions

- In Fig. 3, why don't you train the model as many steps as FermiNet did? - There are other works on generalized wave functions mentioned in the related-work section, such as PESNet. Why aren't they compared in the experiments? - If I understand correctly, for the large molecules in Fig. 5, none of the neural wave functions outperforms CCSD(T) baseline?

Rating

6

Confidence

3

Soundness

4

Presentation

4

Contribution

4

Limitations

The authors provided a section describing the limitations of the current work.

Summary

This paper proposes NeurPf, a novel approach that replaces the standard determinant structure with a Pfaffian-based structure that allows systems of varying sizes to be represented with a single neural wave function. The key idea of the new ansatz is that given a large enough skew-symmetric matrix $A$, we have $\text{Pf}(BAB^\top) = \det(B) \text{Pf}(A)$ for invertible matrix $B$, allowing anti-symmetry to be broadcasted from the neural orbitals to the final output. This makes it possible to set the number of output orbitals $N_o$ into a fixed value that is irrelevant to the system size $N_e$ (so long as $N_e \leq N_o$). Several details on implementing the NeurPf, including architecture selection, envelops, and computation, are discussed. Further, the authors ran experiments on the second-row atoms and small molecular systems to verify the effect of the proposed method.

Strengths

- I think the major contribution, i.e., using Pfaffian to overcome the size consistency issue for varying systems, is quite novel and compelling. It is not only a direct generalization of the Slater determinant (which means that it can be directly applied to most existing architectures), but it allows systems with varying sizes to be represented with one set of parameters. - The paper is well-written and organized, making it easy to follow and understand. - The experiments that the authors consider, while not large enough systems (will be discussed later), indeed demonstrate the potential of the proposed method. I especially like the joint training experiments on all atoms of the second row, which serve as solid evidence that the proposed NeurPf ansatz can be used for multiple systems.

Weaknesses

I think the following weaknesses are important to be addressed for the sake of a strong paper: - First, the authors emphasize the generalization ability of NeurPf, but it looks like all experiments they conduct are a sense of joint training (which I agree is still good to pursue). That said, the authors demonstrate the potential of training one network for multiple models but have not shown that the trained model can be generalized to unseen (but relevant) systems (probably, I admit, with some fine-tuning required). I think it will be necessary for the authors to demonstrate the capacity of generalization since the advantage of NeurPf is exactly to represent multiple systems together. For instance, training on 7/8 of the second row systems and generalizing to the rest, training on the ionized systems of some of the atoms and generalizing to the ionized systems of the rest, etc. - To me, NeurPf is, instead of a separate algorithm, more of an ansatz modification that can be applied to existing ansatzes, e.g., Ferminet, PsiFormer, LapNet, Moon, and Globe. Applying the method to all those architectures empowers them to be trained jointly on varying systems, as well. Unfortunately, the results of this combination are lacking, and the comparison the authors presented is only between the proposed method applied by Gao et al. 2023a. I think is will be important if the authors show that 1. If we apply NeurPf to Ferminet, Psiformer, LapNet, Gao et al. 2023a, and TAO, then the joint training performance is similar to separate training, while the efficiency is much better. 2. For Fig 5, the results of all existing methods other than TAO should also be presented. 3. Comparison between NeruPf on different architectures to demonstrate which one is / is not compatible with the proposed architecture. - Can the authors add a concrete analysis of the computation efficiency of NeurPf? Specifically, how is it compared to a fixed-size Slater determinant (if $N_e = N_o$)? In order to train all systems together, we have to use the largest size (which implicitly increases the computation cost of the smaller systems). How will this influence the overall efficiency? Some plots or tables (instead of a number in texts) are preferred. - The authors should apply NeurPf on metal atoms (whose structures are complex but similar, and whose ionized energies are important to compute) to see how well it goes. For instance, Training on Na, Mg, Al, K and Ca. These systems are within 20 electrons (smaller than C(NH)_2), and should be feasible to train. The intuition is that since metals share similar electron organization structures to some extent, the proposed method should be able to capture the similarity and hence outperform separate training.

Questions

This is irrelevant to my rating, but right now the number of dimension $N_o$ must be larger than the size of the maximum systems $N_e$. What do you think we should do as the system size $N_e$ goes up to 100, e.g. in Psiformer and LapNet paper?

Rating

6

Confidence

5

Soundness

4

Presentation

4

Contribution

3

Limitations

Irrelevant.