Library

Infusing Self-Consistency into Density Functional Theory Hamiltonian Prediction via Deep Equilibrium Models

In this study, we introduce a unified neural network architecture, the Deep Equilibrium Density Functional Theory Hamiltonian (DEQH) model, which incorporates Deep Equilibrium Models (DEQs) for predicting Density Functional Theory (DFT) Hamiltonians. The DEQH model inherently captures the self-consistency nature of Hamiltonian, a critical aspect often overlooked by traditional machine learning approaches for Hamiltonian prediction. By employing DEQ within our model architecture, we circumvent the need for DFT calculations during the training phase to introduce the Hamiltonian's self-consistency, thus addressing computational bottlenecks associated with large or complex systems. We propose a versatile framework that combines DEQ with off-the-shelf machine learning models for predicting Hamiltonians. When benchmarked on the MD17 and QH9 datasets, DEQHNet, an instantiation of the DEQH framework, has demonstrated a significant improvement in prediction accuracy. Beyond a predictor, the DEQH model is a Hamiltonian solver, in the sense that it uses the fixed-point solving capability of the deep equilibrium model to iteratively solve for the Hamiltonian. Ablation studies of DEQHNet further elucidate the network's effectiveness, offering insights into the potential of DEQ-integrated networks for Hamiltonian learning. We open source our implementation at https://github.com/Zun-Wang/DEQHNet.

cs.LGAdvanced Graph Neural NetworksMachine Learning in Materials ScienceQuantum many-body systemsChemistryComputer Science
View graphSource

Paper

References (54)

01Neural-network Density Functional Theory Based on Variational Energy Minimization.Yang Li, Zechen Tang, Zezhou Chen et al.2024 · Physical Review Letters · 18 citations In Library
02Self-Consistency Training for Density-Functional-Theory Hamiltonian PredictionHe Zhang, Chang Liu, Zun Wang et al.2024 · International Conference on Machine Learning · 14 citations In Library
03Self-consistent Validation for Machine Learning Electronic StructureGengyuan Hu, Gengchen Wei, Zekun Lou et al.2024 · arXiv.org · 2 citations In Library
04Reducing the Cost of Quantum Chemical Data By Backpropagating Through Density Functional TheoryAlexander Mathiasen, Hatem Helal, Paul Balanca et al.2024 · arXiv.org · 6 citations In Library
05DeepH-2: Enhancing deep-learning electronic structure via an equivariant local-coordinate transformerYuxiang Wang, He Li, Zechen Tang et al.2024 · 28 citations In Library
06Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passingYusong Wang, Tong Wang, Shaoning Li et al.2024 · Nature Communications · 135 citations
07TorchDEQ: A Library for Deep Equilibrium ModelsZhengyang Geng, J. Kolter2023 · arXiv.org · 25 citations In Library
08EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree RepresentationsYidong Liao, Brandon Wood, Abhishek Das et al.2023 · International Conference on Learning Representations · 365 citations In Library
09QH9: A Quantum Hamiltonian Prediction Benchmark for QM9 MoleculesHaiyang Yu, Meng Liu, Youzhi Luo et al.2023 · Neural Information Processing Systems · 44 citations In Library
10Efficient and Equivariant Graph Networks for Predicting Quantum HamiltonianHaiyang Yu, Zhao Xu, X. Qian et al.2023 · International Conference on Machine Learning · 58 citations In Library
11Long-Short-Range Message-Passing: A Physics-Informed Framework to Capture Non-Local Interaction for Scalable Molecular Dynamics SimulationYunyang Li, Yusong Wang, Lin Huang et al.2023 · International Conference on Learning Representations · 32 citations In Library
12General framework for E(3)-equivariant neural network representation of density functional theory HamiltonianXiaoxun Gong, He Li, Nianlong Zou et al.2022 · Nature Communications · 169 citations In Library

Scroll for more · 38 remaining

Similar papers

Self-Consistency Training for Density-Functional-Theory Hamiltonian PredictionHe Zhang, Chang Liu, Zun Wang et al.2024 · arXiv (Cornell University) · 14 citations In Library
A Density Functional Tight Binding Layer for Deep Learning of Chemical Hamiltonians.Haichen Li, Christopher R. Collins, Matteus Tanha et al.2018 · Journal of Chemical Theory and Computation · 99 citations In Library
Deep-learning density functional theory Hamiltonian for efficient ab initio electronic-structure calculationHe Li, Zun Wang, Nianlong Zou et al.2021 · Nature Computational Science · 242 citations In Library
Deep learning and density-functional theoryK. Ryczko, David A. Strubbe, Isaac Tamblyn2018 · Physical Review A · 88 citations In Library
Efficient and Equivariant Graph Networks for Predicting Quantum HamiltonianHaiyang Yu, Zhao Xu, X. Qian et al.2023 · International Conference on Machine Learning · 56 citations In Library
Efficient and Scalable Density Functional Theory Hamiltonian Prediction through Adaptive SparsityErpai Luo, Xinran Wei, Lin Huang et al.2025 · International Conference on Machine Learning · 12 citations In Library
Quantum Deep Equilibrium ModelsP. Schleich, Marta Skreta, L. B. Kristensen et al.2024 · Neural Information Processing Systems · 6 citations In Library
Enhancing the Scalability and Applicability of Kohn-Sham Hamiltonians for Molecular SystemsYunyang Li, Zaishuo Xia, Lin Huang et al.2025 · International Conference on Learning Representations · 15 citations In Library
Quantum deep field: data-driven wave function, electron density generation, and atomization energy prediction and extrapolation with machine learningMasashi Tsubaki, T. Mizoguchi2020 · Physical Review Letters · 45 citations In Library
General framework for E(3)-equivariant neural network representation of density functional theory HamiltonianXiaoxun Gong, He Li, Nianlong Zou et al.2022 · Nature Communications · 163 citations In Library

Peer review

Accept (poster)NeurIPS 2024 · via OpenReview
Reviewer mPEr5/10 · confidence 2/52024-07-06

Summary

Paper presents DEQH model which is a Deep Equilibrium Model to solve quantum Hamiltonians. DEQH uses the deep equilibrium model because it converges to a fixed point, which matches the self-consistent nature of Hamiltonian solving. Paper presents architecture based on the QHNet, results and comparison with QHNet for the MD17 and QH9 datasets, and convergence and ablation study.

Strengths

Paper addresses a significant problem of computational chemistry. Paper is original and novel in integrating deep equilibrium model with Hamiltonian solving (combining fixed point convergence and self-consistency property makes sense). Paper has good set of experiments comparing with an alternative state of the art model and reaches better results than other model.

Weaknesses

It seems there are differences between DEQH and QHNet, including the input of the predicted H, but the differences are not clearly highlighted in the paper. this makes the comparison harder to interpret. The result of H, Psi errors being low while epsilon errors are high for DEQH is strange. What is the meaning of H, epsilon, and Psi, and why is the epsilon error higher in many cases? No error bars on tables and ablation study, makes results harder to interpret.

Questions

see above.

Rating

5

Confidence

2

Soundness

2

Presentation

2

Contribution

3

Limitations

A limitation I see is that DFT ground truth calculation is still used in the loss. Since DEQH model is a Hamiltonian solver, it would be nice to see how it performs using a self-consistency loss. It seems that avoiding having to run many DFT calculations would be a big improvement, whereas the current results of paper show DEQH gets lower error results than previous model but still needs DFT data and requires 1.5x training time.

Reviewer mPEr2024-08-12

Thank the authors for response. I still think the ablation study is hard to interpret, but I appreciate the additional comparison with other benchmarks. I have increased score accordingly

Authorsrebuttal2024-08-13

We are pleased that our response has satisfied you and are very grateful for the increased score. As outlined in our rebuttal, the DEQH functions as a Hamiltonian solver, learning the iterative solving process $H^* = f(H^*, Z, R)$, while the QHNet acts as a predictor with the formula $H = f(Z, R)$. In practice, if the dataset includes overlap matrices, the network input can also incorporate this information, treating it similarly to the Hamiltonian by transforming it into equivariant node features (even if it's absent from the dataset, the calculation of the overlap matrix is quite straightforward and can be generated during data preprocessing). The overlap matrix offers a wealth of detailed information and can be computed easily with $Z$ and $R$. Accordingly, the equations for the DEQH and the modified Hamiltonian predictor become $H^* = f(H^*, Z, R, S)$ and $H = f(Z, R, S)$, respectively, where $S$ is the overlap matrix. In line with this, we conducted several experiments in our ablation study, testing the QHNet ($H = f(Z, R)$), QHNet w/ S ($H = f(Z, R, S)$), DEQHNet ($H^* = f(H^*, Z, R, S)$), and DEQHNet w/o S ($H^* = f(H^*, Z, R)$) on the Uracil dataset. The results depicted in Figure 4 indicate that: * Incorporating the overlap matrix as an input leads to a lower Mean Absolute Error (MAE) in predicting the Hamiltonian (DEQHNet's Hamiltonian MAE is lower than DEQHNet w/o S, and QHNet's Hamiltonian MAE is higher than QHNet w/ S). Furthermore, the inclusion of the overlap matrix results in higher similarity in the orbital coefficients, suggesting that the overlap matrix is beneficial for the network's learning of the Hamiltonian and also supports our experimental hypothesis that the overlap matrix provides orbital information, which in turn improves the results for the orbital coefficients. * Regardless of the presence of the overlap matrix in the network's input, the DEQH's Hamiltonian MAE is consistently lower than that of the QHNet-based Hamiltonian MAE, indicating that the DEQH model benefits from the introduction of self-consistency. We will ensure that all new experimental results are included in the manuscript and will promptly refine our paper as soon as we are permitted a polish for the final version. We deeply appreciate your valuable feedback.

Reviewer DtX87/10 · confidence 2/52024-07-12

Summary

The paper introduces a novel neural network architecture DEQH, extending deep equilibrium models (DEQs) to improve predictions of quantum Hamiltonians. The architecture constrains solutions to ensure self-consistency of the Hamiltonian, thereby improve generalization capability and test accuracy.

Strengths

Constraining the network to obey self-consistency is natural thing to do and matches the physical constraints of the problem at hand, without the need for costly DFT calculations. The paper proposes an elegant parameterization. Further, it provides an extensive empirical analysis showing the practical benefits of the proposed approach on several datasets.

Weaknesses

As mentioned, DEQH model acts fundamentally as a solver, iteratively determining the Hamiltonian with fixed point iterations. It is not directly clear to me why this would necessarily be better, or how this compares, to methods that integrating frameworks that also ensure self-consistency. Why would we expect to be better both computationally and in terms of performance, or is there a clear trade-off?

Questions

How do we expect DEQH to compare to methods that ensure self-consistency through integration? Is there are trade-off between performance and computational cost, or do we expect the method to be better overall. Why?

Rating

7

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The paper proposes a novel way to embed self-consistency into neural network architectures for quantum Hamiltonian prediction and provides empirical evidence of benefits.

Reviewer Tf4v4/10 · confidence 2/52024-07-13

Summary

The authors introduce DEQH model, which combines deep equilibrium models with existing ML models to predict quantum Hamiltonian, and the author adopt QHNet as the backbone and further develop DEQHNet. The authors evaluate the proposed method on benchmarks MD17 and QM9, the results show some effectiveness.

Strengths

1. The idea of incorporating the DEQs is interesting, which have some kind of intrinsic principles as the authors demonstrated in the paper. 2. When compared with QHNet, the DEQHNet shows advantages.

Weaknesses

1. Though I am relatively familiar with ML for Hamiltonian prediction and have gained some knowledge of DEQ, it is still not easy for me to understand the main part of this paper. 2. This work only compare with one baseline named QHNet, there are some other works which can also be baselines [1] [2] [3]. 3. Source code is not available. [1] Unke O, Bogojeski M, Gastegger M, et al. SE (3)-equivariant prediction of molecular wavefunctions and electronic densities[J]. Advances in Neural Information Processing Systems, 2021, 34: 14434-14447. [2] Gong X, Li H, Zou N, et al. General framework for E (3)-equivariant neural network representation of density functional theory Hamiltonian[J]. Nature Communications, 2023, 14(1): 2848. [3] Wang Y, Li H, Tang Z, et al. DeepH-2: Enhancing deep-learning electronic structure via an equivariant local-coordinate transformer[J]. arXiv preprint arXiv:2401.17015, 2024.

Questions

1. Why is the model performance so much worse than the baseline in terms of energy? Has the author explored the reasons? 2. Is the method proposed by the author extensible and can it be applied to other existing ML models?

Rating

4

Confidence

2

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors have discussed the limitations.

Authorsrebuttal2024-08-10

Thank you for sharing your further comment. Nevertheless, we are afraid that your comment is based on a factual misunderstanding. The fact is, DEQHNet outperforms QHNet in all cases, including ethanol and malonaldehyde, with the only exception of water due to limited available labels (500; other molecules have 25,000 labels) as we have explained in the paper. Such results indicate that DEQHNet improves upon existing end-to-end Hamiltonian prediction methods universally. We are not sure about the specific reason why you have the misunderstanding, but we would like to stress that the primary comparing metric should be Hamiltonian MAE, which directly measures the prediction accuracy of the models. Other metrics, i.e., orbital energy and orbital coefficients, are derived quantities from the Hamiltonian. As we mentioned in the rebuttal, the prevailing MAE metric for Hamiltonian has been found not perfectly aligning with metrics of these derived quantities, which has also been observed in the comparison between QHNet and PhiSNet. In the appendix of our paper and in the rebuttal, we have explained why our method performs well in terms of Hamiltonian error and orbital coefficients. We also pointed out that the reported orbital energy falls within the error range of the current Hamiltonian. Specifically, in Section A.9 of the Supplementary Material and the Question 1 section of the rebuttal, we discussed this discrepancy in detail. We randomly selected a molecule, added a symmetrized Gaussian noise matrix (Hamiltonian matrices need to be symmetric), and solved the corresponding generalized eigenvalue equation. As shown in Fig. 5, as the noise on the Hamiltonian increased, the error range of the orbital energy also increased. The orbital energy errors we reported for both QHNet and DEQHNet on the MD17 and QH9 datasets fall within the range shown in the figure, suggesting that the experimental results are reasonable. Additionally, we can provide further evidence from the QHNet paper, which observed a similar phenomenon. In their study, the PhiSNet achieved a Hamiltonian error of only 15.67x10$^{-6}$ Hartree on water, but the orbital energy error was "ten times worse than that by QHNet (33.76)", approximately 330x10$^{-6}$ Hartree. This observation implies that only when the Mean Absolute Error (MAE) of the Hamiltonian is sufficiently small can we ensure that the corresponding orbital energy deviation is also small. In the comparison between QHNet and PhiSNet, we observed similar situations. This somewhat suggests that using existing Hamiltonian error measurements cannot definitively reflect the errors in the eigenvalues. These evidence suggests that it is a nontrivial and long-standing challenge in the domain of Hamiltonian prediction to design a proper metric on the matrix space so that reflects the metric of derived quantities e.g. energy, for which we will investigate in future work. Finally, concerning your comment that "the presentation of the paper needs further improvement," we have already stated in our rebuttal that "We will also add further explanations about the integration of these two components and how this combination contributes to the overall model. We hope that these revisions will make our paper more comprehensible to readers with various backgrounds." The PDF we provided in the global rebuttal is merely an initial visualization, as the current NeurIPS review process only permits a one-page PDF that includes images and tables but no text.

Reviewer eJGc6/10 · confidence 2/52024-07-30

Summary

This paper introduces the DEQH (Deep Equilibrium Quantum Hamiltonian) model, which integrates Deep Equilibrium Models (DEQs) for predicting quantum Hamiltonians. By incorporating DEQs, the model captures the self-consistency of Hamiltonians without needing iterative Density Functional Theory (DFT) calculations during training, enhancing computational efficiency. DEQHNet, a specific implementation, demonstrates improvements in prediction accuracy on the MD17 and QM9 datasets. The model acts as both a predictor and solver, iteratively refining the Hamiltonian to achieve self-consistency. Ablation studies further validate the effectiveness of this approach.

Strengths

1. The DEQH model proposed eliminates the need for iterative DFT calculations during training, which reduces computational overhead. 2. The DEQHNet model demonstrates improved accuracy in predicting Hamiltonians on the MD17 and QM9 datasets. 3. The model inherently captures the self-consistency required for accurate Hamiltonian prediction. 4. This paper includes ablation studies to analyze the contribution of different components of the model. 5. The paper shows quick convergence of the DEQHNet model.

Weaknesses

1. The presentation of the paper could be improved. In this paper, the integration of Deep Equilibrium Models (DEQs) with the Hamiltonian solver is presented in a quite technical way. Additional visual illustrations and diagrams could help clarify the workflow of the DEQH model and its components. 2. This paper primarily compares DEQHNet with QHNet. Benchmarking with additional methods could help better evaluate the results.

Questions

What are the runtime and memory usage for DEQHNet compared to previous methods? What are the theoretical assumptions underlying the DEQH model, and how do they impact the generalizability of the results?

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations are discussed.

Reviewer Tf4v2024-08-10

Thank you for your efforts and detailed response. However, I still have some concerns about the additional experiments: DEQHNet performed poorly on water, not even surpassing QHNet; its performance on ethanol was also not better than QHNet; and on Malonaldehyde, it showed no advantage compared to the baseline. It only had some advantage on uracil. Additionally, although the authors provided visualizations to demonstrate the method, I still believe that the presentation of the paper needs further improvement (the Reviewer eJGc also mentioned this point). In summary, although the idea of DEQHNet seems interesting, its presentation and experimental performance are not convincing enough, so I will maintain my original rating.

Reviewer eJGc2024-08-11

Thanks a lot for taking the time and effort to answer my questions. I would be considering raising the score in the next two days.

Authorsrebuttal2024-08-12

We are glad to know that our response is satisfactory to you. We will make sure to include all the new experimental results in the manuscript and improve the presentation as soon as we are permitted a polish for the final version. We are grateful for your valuable feedback.

Reviewer DtX82024-08-13

I thank the author’s for the further explanation and clarifications. I keep my recommendation for acceptance with the score of a 7, conditioned on including discussions and additional experiments from the other rebuttals in the final manuscript.

Authorsrebuttal2024-08-13

We are delighted that our response has met with your satisfaction. We will make sure to incorporate all of the latest discussions and experimental results into the manuscript and enhance its overall presentation promptly once the opportunity for a final polish is granted. Your constructive feedback is highly appreciated.

Authorsrebuttal2024-08-14

We are very grateful for the increased score and deeply appreciate your valuable feedback.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC