Towards Symmetry-Aware Generation of Periodic Materials

Summary

The problem of generating periodic materials with deep models is considered, focusing on capturing their unique symmetries. SyMat, a novel material generation approach, is proposed, using a variational auto-encoder to generate atom types, lattice lengths, and angles. A score-based diffusion model with a symmetry-aware probabilistic model is employed for generating atom coordinates, demonstrating promising performance in random generation and property optimization tasks.

Strengths

- Clear presentation - Good structure - Fair comparison to prior work (CDVAE) - Important application (materials discovery) - Convincing empirical results - Clear limitations section

Weaknesses

- Contributions to ML method development minor

Questions

- Eq. 2: Why did you decide to factorize $p(A, L)$ into $p(A) * p(L)$. I would have guessed that coupling $A$ and $L$ is important. - Did you consider tackling the permutation symmetry in $[\ell, \phi]$, where permuting $\ell_1$ with $\ell_2$ leaves $\phi_{1,2}$ unaffected (or does it?).

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

As the authors comment, diffusion models are computationally intensive. I don't consider that a direct problem of the paper, though.

Summary

this paper proposes a method to enforce required symmetric properties in output generated by diffusion models. this method is applied to the problem of generation of new materials with desired symmetric properties. the method comprise of VAE for the generation of atom numbers, A, and lattice structures, L, while diffusion is used for the generation of coordinates, P. The main assumption in this work is that the probability of a molecule does not change under permutation/rotation+translation.

Strengths

an insightful approach for incorporating required symmetric properties. Figure 2 helps to understand the entire method.

Weaknesses

the assumption on equiprobable molecules, P(M)=P(M'), is not true in general. Though it can work in some cases. the assumption that "type vector A" is independent on "lattice matrix L" in the generation of a new material sounds heuristic. could you please elaborate of this assumption and its limitations?

Questions

could you please explain why P(M) = P(M') in Section 3.1? There are inter-atom, A, interactions, which may affect the properties of a molecule, including its energy etc. The probability to find M' in nature can be significantly different from that of M. So, I am not sure that P(M)=P(M') under 'Permutation' and/or 'Rotation and translation' of A and P is a reasonable assumption in general. If I am wrong, please explain. Otherwise, please explain the limitations of this assumption. Could you please explain "Since the structures of atom types and lattices are relatively simple.."? Is not it a combinatoric problem? What does it mean simple in this context? How is this simplicity related to VAE? VAE has quite a lot of drawbacks. Why do you use different approaches for A+L and for P, VAE, and Diffusion, respectively? You mention that Diffusion is more powerful than VAE, so it can not 'harm' the quality of A+L in comparison to VAE? Why to complicate? line 187: "Afterwards, the decoder model uses zA and zL to reconstruct the exact $c$, $\ell$...". VAE usually has a reconstruction error. How do you guarantee "exact reconstruction"? Can you explain why "our method cannot be applied to non-periodic materials"? What are the possible directions for the acceleration of "atom coordinate generation"?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

the assumption about the equiprobable molecule generation P(M) = P(M'), is limiting in my option. the authors adequately address the limitations in Section 3.5.

Summary

This paper proposes SyMat, a new approach to generate materials that capture physical properties of perodic structures. This method generates atom types, lattice matrix of the unit cell, and then atom positions, all while maintaining invariance to permutations, rotation, translation, and periodic transformation. They use a VAE (with SphereNet as the encoder and MLPs as decoder) to generate the elements and lattice vectors, and score-based diffusion model to generate the atom positions. They evaluate SyMat in random generation and property optimization tasks.

Strengths

- This paper tackles an important problem of generating periodic materials, useful for the field of materials discovery given that datasets of known materials for inference are limited - Good techniques of how to frame the prediction task in a symmetry-aware way - Good comparison to related work and including several of them as baselines - Generally strong results, with SyMat performing best or second best in nearly all metrics

Weaknesses

- I don't fully agree with not needing to evaluate the ability of the model to reconstruct materials from their latent representations. 1: even if the goal is to generate novel materials, we still want to ensure that the latent representation is valid and useful, and reconstruction of known materials seems to be a good way of validating the latent representations. 2: Agree that you can't use absolute coordinates for evaluation, but you should be able to evaluate it based on interatomic distances (or perhaps at least something like graph connectivity). - Is there some way to evaluate diversity in the random samples generated? Also, ideally it would be great to see if the generated materials are stable (e.g. able to converge from some relaxation), but perhaps that's out of scope.

Questions

- For success rate of property optimization, "the rate of decoded materials whose properties are in the top 5%, 10%, and 15% of the energy distribution in the dataset", what is "the dataset" referring to - is it just some energy threshold determined by the Nth percentile of the test set? Would it be helpful to report the distribution of energies genereated in addition to success rate at thresholds? - How is the energy of the generated materials obtained? Is it verified by something like DFT? i.e. to make sure that the model isn't cheating by reporting artificially low energies

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

- The authors acknowledge limitations such as model inference speed and that the work is limited to task of periodic structures

Summary

This paper introduces a deep learning based method for the automatic generation of periodic molecules. At the core of their method lies a variational autoencoder and a score-based diffusion models, which generate lattice lengths and angles, and atom coordinates, respectively. Theoretical derivations show that their method is invariant to symmetric transformations and quantitative validation show that the proposed method can be used for optimization problems. I have read the rebuttal. I am more positive about this submission after the rebuttal and reading the other reviews. The authors have addressed my concerns regarding empirical analysis of limitations, and the quality of writing and figures.

Strengths

- The paper is overall well written and easy to follow. - Extensive validation is provided, with comparisons with previous work across a variety of metrics and with ablation studies of the components of their own method. - Invariance to symmetric transformations is achieved through theoretical derivations, which makes the model more robust to these transformations than data augmentation-based methods. - The provided supplementary material is quite solid and shows very interesting results and ablations. - The paper and supplementary material contain enough detail to help reproducibility of the results.

Weaknesses

- I believe the paper could do a better job at motivating the need for modelling symmetry in molecule generation. I am no expert in this topic and I lacked some motivation onto why the problem is salient. - The figures in the paper could be improved, particularly Figure 2 could benefit from better illustrations. - Some design choices could be motivated better, particularly related to the model design. - The paper needs, in my opinion, more qualitative results like those provided in Figure 3 in the supplementary material. - The potential limitations of SyMat are not thorougly analyzed.

Questions

- Will code be provided? - Why is denoising score matching "more effective to train" the generative model?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

In my opinion, the authors could provide more discussion of the limitations of their work.

Response to rebuttal

I am glad my comments were useful! Thank you for your detailed response. I am also glad that code will be shared I suggest you include a summary of your excellent responses to my Q3 and Q7 on the paper, as it should help motivate the design choices, which will increase the potential impact of the paper.