Grassmann Manifold Flows for Stable Shape Generation

Summary

This paper proposes a normalizing flow approach applicable to generate data on Grassmann manifold. The proposed method utilize the fact that Grassmann manifold is a quotient space of Stiefel manifold on which numerical calculation is feasible. Specifically, the method first constructs the flow from vector field and then transform a probabilistic density using the flow. The vector field is learned through a neural network. A prior probabilistic density is defined as a matrix-variate Gaussian distribution on Grassmann manifold. The experiments are conducted on three different types of datasets among which the first one is synthetic and the last one is molecular point clouds. The experimental results show that the proposed GrCNF can generate the distribution on Grassmann manifold.

Strengths

This paper aims to resolve the problem of shape generation when the high-dimensional shapes intrinsically embed in a low-dimensional manifold, e.g., Grassmann manifold, using normalizing flow (NF). Extending NF from common Euclidean space to non-linear manifold is novel and meaningful, as the manifold hypothesis assumes that most high-dimensional data usually embeds in a low-dimensional manifold. The theoretical part looks sound. Experiments under three different settings verify the effectiveness of the proposed GrCNF to learn distributions on Grassmann manifold.

Weaknesses

The main weakness (also my concern) is, as also mentioned by the authors in Line 325 - 328, whether the proposed GrCNF is applicable to high-dimensional data? E.g., point clouds containing thousands of points? If yes, it will make the paper very strong in both theoretical and application aspects. (I also want to know what real-world high-dimensional data embeds in Grassmann manifold?) If not applicable to high-dimensional data, it will weaken the paper.

Questions

In general: 1. What's the meaning of "stable shape" in the title? Why does the generation process is stable? Method parts: 2. In Figure 1, what's the relation between T^v_YSt(k, D) and T^h_YSt(k, D) and why do we use T^h_Y, not use T^v_Y? 3. In Line 106, it mentions that "we use the representative Y ...", so how to choose the representative Y? 4. In Proposition 1, what's the relation between a vector field and a flow? I.e., how to compute a flow F_{X, T} given a vector field X? 5. Why do we need retraction in Eq.(4) in Proposition 3? 6. In Section 5.3 and Figure 2. does the output (through a neural network) is ensured in the tangent space of T^h_YSt(k, D) theoretically? 7. In Section 5.4, what's the goal of the ODE solver? Could you use one or two sentences to summarize it. Experiment parts: 8. In the experiment in Section 6.1, I'm not sure why the spiral (and swissroll and circle ...) are 1-dimensional data in R^3, as they contain x and y two coordinates. 9. In the experiment in Section 6.2 and 6.3, is it possible to visualize the generated data, similar to Figure 3. Implementation parts: 10. As the method is complicated in theory, will the code be released?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The authors mentions the two limitations of the proposed method in Line 324 - 328.

Summary

This paper presents the application of continuous normalizing flow (CNF) to the Grassmann manifold for shape generation. The authors establish the theoretical foundations for learning distributions on the Grassmann manifold using CNF, enabling the generation of stable shapes. The proposed approach offers a promising generative model capable of generating subspace data.

Strengths

The study represents the first exploration of applying a continuous normalizing flow (CNF) to the Grassmann manifold for shape generation. The experiments conducted demonstrate the effectiveness of the proposed approach in generating promising shapes, particularly subspace data with low dimensions.

Weaknesses

I have concerns regarding the significant contribution of the paper to the domain, given the existence of [51]. To address these concerns, it would be beneficial for the authors to include a comparative analysis with [51] in the experiments to demonstrate any potential superiority of the proposed method. Furthermore, it is not clear how the proposed approach technically differs from the Stiefel-based ODE [7]. To provide a clearer understanding, it would be helpful if the authors could provide an intuitive illustration of the transfer from the Stiefel-based ODE to the Grassmann-based ODE. Additionally, the paper's validation is limited to low-dimensional data, and there is a lack of empirical study on real-world, high-dimensional data such as point clouds. As a result, the current validation may not be sufficiently convincing to establish the effectiveness of the proposed approach in practical scenarios. Conducting experiments on high-dimensional data, such as point clouds, would enhance the credibility and generalizability of the proposed method.

Questions

There should indeed be a comma rather than a period before finishing the equation, as seen in Eq. 8 on lines 232-233. Regarding the performances for different D and k values on DWR, LJ13, and QM9, it would be valuable for the paper to provide an analysis and evaluation of the proposed method's performance in these scenarios. This would enhance the understanding of the method's effectiveness across different datasets and settings. The reason why the proposed GrCNF outperforms the counterpart methods in Table 1 should be clarified in the paper. It would be beneficial for the authors to provide an explanation or analysis that highlights the specific advantages or characteristics of the proposed method that contribute to its superior performance. Regarding the stability of shape generation in the proposed method compared to other methods, the paper should provide a clearer explanation to address this question. It would be helpful for the authors to discuss the specific mechanisms or techniques employed in the proposed method that lead to more stable shape generation results. The feasibility and necessity of bringing invertibility to normalizing flows could be an interesting aspect to explore. It would be valuable for the authors to discuss the potential benefits and drawbacks of incorporating invertibility into the proposed method, considering the existing literature and the specific requirements and objectives of the task at hand.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

The paper does not provide a thorough comparison with existing methods, such as [51], to demonstrate the superiority of the proposed approach. A comparative analysis would have helped assess the advancements of the proposed method over previous work. The paper lacks empirical studies on real-world, high-dimensional data, such as point clouds. This limitation restricts the generalizability and applicability of the proposed approach to practical scenarios, as the effectiveness of the method has only been demonstrated on low-dimensional data. It is not adequately explained how the proposed approach technically differs from the Stiefel-based ODE [7]. A clearer illustration of the transfer from Stiefel-based ODE to Grassmann-based ODE would have provided a better understanding of the method's technical contributions. While the paper claims that the proposed GrCNF outperforms counterpart methods, it does not provide a comprehensive explanation for this superiority. It would be beneficial to discuss and analyze the specific factors or characteristics of the proposed method that contribute to its improved performance.

Summary

This paper extends the previous Riemannian CNF to subspaces, i.e., Grassmann Manifold Gr(k,D), and tackle stable shape generation with nice transformation-invariant property. They derive the first and full mathematical formulation for learning invariant densities and flow on a Grassmann manifold. They successfully bring the complete model to life by using Stiefel Manifold to help construct the vector field effectively, with adapted ODE solver on Gr(k, D). They have also shown the superior performance of the proposed method on both toy and well-adopted public benchmarks compared to several existing seminar works.

Strengths

[**originality**] the proposed work is the first one to explore continuous normalizing flow on subspace on the Grassmann Manifold. They not only creatively combine the success of quotient manifold for shape analysis, and the recent Riemannian CNF, but also contributed an elegant way of constructing the vector field through the connection to Stiefel manifold with horizontal lift. [**quality & clarity**] The writing quality is over the bar, with clean explanation on the motivation, intuition, and design. As a person who is not that familiar with manifold theory, I found that the mathematical derivation is straight-forward to understand. They have presented complete and detailed propositions and proofs throughout the main paper and supp. [**significance**] The proposed framework is a pioneer work on exploring continuous normalizing flow on subspaces with nice equivariance-preserving property. Compared to existing baselines, this method is achieving SOTA performance on simple shape generation. It has great potential to benefit 3D molecules design, other high-dimensional geometry data generation including common objects.

Weaknesses

1. From Figure 2, all that has been showing is on the Stiefel Manifold, without showing how it is connected to the target Grassmann manifold, which is confusing and not informative. Without this connection, I am not clear how the nice O(k)-invariant output v_out is achieved. The author might want to update the figure, or provide a better clarification; 2. As the author also mentions, it will make the proposed work more significant if showing experimental results on real-world object shapes. More importantly, I am interested in knowing the major challenges to use the developed framework to handle such object shape data; 3. On the public benchmarks, the quantitative numbers are better than the baselines, it will make the comparison more complete if we can further show the visualizations of the generated samples;

Questions

1. I am also wondering about the computation overhead, besides the expensive ODE solver, how many steps does it take to generate the final sample from a Gaussian density? 2. When comparing with the baselines, the proposed method will instead take the orthonormalized data, I am wondering whether this is a fair comparison setting.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

NA

Summary

The paper presents a novel concept: Continuous Normalizing Flows on a Grassmann Manifold (GrCNF), a model that presents a more efficient and accurate way to capture and represent the geometric features of data. The unique proposition of GrCNF is its application of the Grassmann Manifold to grasp data equivariance, a key aspect that delineates the transformation response of data. GrCNF's efficacy is demonstrated via several experiments. The results affirm its capability to accurately generate data that depict complex geometric shapes, and outperform current Normalizing Flow models in metrics like log-likelihood and ELBO for diverse test datasets.

Strengths

+ GrCNF is a novel model to bring continuous normalizing flows (CNF) into the field of Grassmann manifolds, expanding its applications to data manifolds with geometric structures. + GrCNF showed superior performance when compared to existing Normalizing Flow models in experiments with different datasets. It not only generated high-quality samples but also effectively learned the distribution on Gr(k, D). + GrCNF is able to handle subspace data, making it a versatile tool for handling various types of complex data.

Weaknesses

1. The use of the Kronecker product at each ODE step is a demanding process, which suggests potential improvements in computational efficiency could be beneficial. This exploration could lead to exciting developments in scalability and applicability. 2. The study primarily focused on lower-dimensional data, and an examination of the GrCNF model's performance with higher-dimensional data could provide additional valuable insights.

Questions

In the artificial texture experiments, the antipodal points were generated using the trained GrCNF. While this approach demonstrates the learning capabilities of the model, it does not guarantee the accurate representation of all antipodal points in different contexts or datasets.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

1. The proposed GrCNF has been tested on a limited variety of data types. Further exploration is required to understand its potential application and effectiveness on different data types, such as image, audio, and textual data. 2. In the experiments, GrCNF is mainly compared with existing normalizing flows methods. The inclusion of other machine learning models, such as generative adversarial networks (GANs) or autoencoders, could have provided a broader perspective on GrCNF's performance.

Summary

This paper is on developing techniques for sampling from probability distributions on Grassmann manifolds and utilizing it to generating shapes. The shapes are associated with the probability distributions on Grassmann and sampling from the distribution generates shapes. The sampling itself it based on flows under vector fields associated with the gradient of potentials (negative logs of pdfs). A large amount of paper is dedicated towards defining vectors fields and flows on a Grassmann manifold assuming one knows how to do it on a Stiefel manifold. Some of these constructions are implemented using deep neural networks. The paper provides some examples of shape and texture generation and compares it to some similar methods.

Strengths

-- The paper is trying to make a connection between a number of distinct and important mathematical quantities: shapes, distributions on Grassmann manifolds, random sampling, gradient flows, etc. -- The geometry part of the paper is solid. The use of horizontal lift to remove the nuisance group is a standard technique in analysis on quotient spaces.

Weaknesses

-- I find this paper somewhat confusing to read. The presentation relies on CNF and Riemannian CNF etc, while these quantities are standard flows under given gradient fields (associated with given pdfs). -- Perhaps the authors can clarify what is different in CNFs from a integral curves or flows under vector fields such as gradient vector fields. See for example, the paper by Liu et al., Optimal Linear Representations of Images for Object Recognition, IEEE TPAMI, May 2004. These are very standard constructions for either Stiefel or Grassmann manifolds. -- The connection with shapes is also unclear. What do the authors mean by a shape? They should explain precisely (preferably in mathematical terms) on how shapes are represented and how sampling from a distribution on a Grassmann becomes generation of a shape.

Questions

See my question listed above. The authors need to clarify in simple layman's terms the big picture of their approach. This can be done without using undefined terminology and stating ideas precisely.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

Yes, they have listed some limitations of the computational steps of their framework.