Modelling brain connectomes networks: Solv is a worthy competitor to hyperbolic geometry!

Summary

This paper studies an interesting problem, that is, embedding the brain connectome to some kind of geometry. To this end, the paper proposes an embedding algorithm based on Simulating Annealing that allows the embedding of connectomes to Euclidean, Spherical, Hyperbolic, Solv, Nil, and also product geometries.

Strengths

An interesting topic about modeling brain connectome networks, which can facilitate the study of brain functions and learning mechanisms.

Weaknesses

The paper may miss some discussion with Euclidean geometry, which the ML community may be more familiar with. For example, in Euclidean geometry, Voronoi tessellation has recently been successfully used to study the mouse brain connectome (in "Network structure of the mouse brain connectome with voxel resolution, Science Advances 2020"). Moreover, there are even more kinds of tessellations beyond Voronoi in Euclidean geometry. Have the authors considered this possibility? On the other hand, network/graph/node embedding has been extensively studied by the deep learning community. I wonder is it possible to apply some graph neural network methods, such as node2vec ("node2vec: Scalable Feature Learning for Networks, SIGKDD") for this problem? The writing and presentation can still be largely improved. For instance, there are multiple typo or grammar issues and it can be better to make the paper more accessible by the machine learning community.

Questions

Can the authors provide more comparisons (and possibly illustrations) to Euclidean embedding? In some cases, the proposed method is never better than the previous state-of-the-art, e.g., on celegans with 3-dim (0 out of 30) and on mouse3 with 3-dim (0 out of 30). Is there any analysis of the failure cases?

Rating

6: marginally above the acceptance threshold

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Summary

This paper presents a simulated annealing based embedding approach for modeling brain connectomes from across different species. This allows modeling of geometries arising from non-euclidean structure- specifically spherical, hyperbolic, solv, nil and other product geometries. The main claims of the paper is that this algorithm is (1) more suitable for finding embeddings in all of considered cases, (2) that resulting solv embeddings providing reasonable performance compared to alternatives and (3) the first of its kind to compare connectome embeddings across different Thurston geometries. Experimental comparisons are performed on simulated and real-world connectomic data. The quality of the resulting embeddings is compared against baseline algorithms on measures such as mean Average Precision (mAP), MeanRank, greedy routing success and stretch

Strengths

The central idea of generalizing connectomic representations to Thurston geometries is novel and very interesting, as is the proposed universal method of constructing such representations. The ideas presented here could be very useful to advancing applications in this domain in the future. Extensive experiments have been performed on simulated and real world connectomes from different species as a comparison, which is a plus.

Weaknesses

1. The style of presentation of content in this manuscript makes it very challenging to follow for readers without the requisite background in topology. (a) For example, section 3 Thurston Geometry introduced a lot of jargon and notation without definition - "universal cover" / $\text{SL}(2,\mathcal{R}). (b) Similarly, comparing the caption description of Fig 1 with the textual description in paragraph 3 of section 3 makes it more difficult and abstract than necessary to understand what the authors are actually trying to convey (A simple fix would be to mark the corresponding points referred to in the text to illustrate the main point). Additionally, the tessalations in the figure are not actually referred to/described in the text using the terminology of the caption, which is very strange. 2. There is little to no background on connnectomics and/or representation learning for connectomics, beyond a few scattered citations. This is rather surprising, since this is a very active field with several works spanning diverse perspectives and approaches from graph theory, statistical models, deep learning, to name a few. The datasets/simulation parameters used in the paper are not described well enough to follow 3. The application aspect of the paper is ill motivated and kind of lost in the emphasis on mathematical explanation. The paper does not do a good job of illustrating why the embeddings are actually helpful for brain connectomics beyond Thurston's conjecture from 1982. This makes it really hard to appreciate why this approach is particularly useful for brain connectomes. In fact, it is very hard to discern what kind of connectomes- functional/structural the embeddings are being applied to. 4. It is not clear how the authors arrive at the likelihood expression in Section 4, which seemingly assumes independence in the pairwise interactions and a functional form for the probability based on the distance measure. Additionally, does the connectome model (V ,E) consider weighted or negative valued edges, as one would obtain from pairwise similarity measure in functional connectomics? 5. The datasets are not described at all in the main paper beyond the references in Table 1. The results in Tables 3 and 4 are really hard to parse and require the reader to go back and forth between the explanation and the table- since the captions are very nondescript, with best performances not highlighted. Additionally, no standard deviation measures have been reported (to quantify variability in either the simulation/and or population)

Questions

It would be great if the authors could work on the following aspects of the paper: (a) Motivating the contribution from the applications perspective (b) Discussing the assumptions made by this embedding approach and why they are suitable/reasonable for this domain - for example, why three dimensional representations are sufficient for high dimensional data (c) Contextualizing the work in light of other approaches used for representation learning (d) Please have a table of notation for easy reference in the appendix and define any abbreviations/notation before usage (e) Would be good to provide insight into the computation complexity of this approach - how computationally expensive is the simulated annealing with Dijkstra's search in Section 4? how does this compare with other approaches? how long does the overall method take to converge? (f) Please provide more details on how algorithmic parameters/ experiments are setup- eg. percentage of data used for computing the embeddings vs independent testing, number of iterations etc

Rating

3: reject, not good enough

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

1 poor

Contribution

2 fair

Ethics concerns

N/A

Summary

This paper studies the visualization of connectomes which are comprehensive maps of neural connections in the brain. This conducts various experiments to compare the use of various geometries including hyperbolic, Solv, Nil, and others on the embedding space.

Strengths

- The study of the visualization of connectomes is important to understand cognitive processes. - The paper conducts a comprehensive study on various geometries.

Weaknesses

- The take-away messages from this paper are too general, not specific, and really useful. As the paper mentions, in many cases, hyperbolic geometry yields the best results, there are other geometries worth consideration, e.g., Solv. Because hyperbolic geometry was studied in the previous work, the add-on Thurston geometries used in this paper cannot yield better results than hyperbolic geometry, and the embedding method used in this paper is not innovative, it is hard for me to see the scientific contributions of the paper. - This paper is possibly more suitable for a journal than ICLR which requires more contributions on machine/deep and representation learning aspect. Moreover, it would be more informative and useful, if the paper comes up with the concrete conclusions regarding what geometries are more suitable for what kinds of connectomes. - The background of hyperbolic geometry has some oversights. For instance, $g^{-}(x,y) = x_1y_1+...+x_ny_n - x_{n+1}y_{n+1}$. Moreover, it is unclear what hyperbolic model the paper talks about (i.e., Lorentz, Klein, or Poincare model).

Questions

- Do you have any conclusions of what geometry should be used for what kinds of connectomes? - For Solv, why do you need to approximate $d(a,b) = d(a,a_1)+ d(a_1, a_2) +... + d(a_k,b)$? What is $d(a_1, a_2)$ in this formulation?

Rating

3: reject, not good enough

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Summary

This paper proposes a new embedding method for connectomes based on Simulated Annealing, which allows embed connectomes to Thurston geometries (Euclidean, spherical, hyperbolic, Solv, Nil, and other product geometries). The proposed method introduces new possibilities in modeling connectomes and is more robust than SOTA, which is crucial. Their experiments demonstrate that the proposed algorithm performs better and finds better embeddings than the SOTA. One of the key findings of this study is that the 3-dimensional hyperbolic geometry produces the best outcomes, while Solv performs as the next best alternative to embed connectomes.

Strengths

1. The paper shows useful results on how Thurston geometries could be helpful in embedding connectomes. 2. The paper is well-written and structured. 3. Overall, this is an essential and comprehensive study with a reasonable amount of experiments that provide very interesting theoretical results. The authors support their theory with empirical results.

Weaknesses

1. Some tables, such as Table 2, lack readability. Including indicators like up or down arrows alongside measurements such as NLL and MAP and highlighting the best outcomes will provide readers with a clear indication of value trends. 2. Some notations and abbreviations need more explanations. It's good to have consistency throughout the paper (while I did not go into the details of all the proofs, the overall sketch and techniques seem correct). 3. At some point, the authors mentioned when it comes to Euclidean geometry, the results are inconsistent. For human connectomes, E^3 outperforms other geometries. What are the possible reasons for such behaviors?

Questions

1. What is the HRG model on page 4, 5th line of 2nd Paragraph? Missing references here. 2. Is there an ablation study on picking M=2000 points for most experiments? 3. Would it be possible to visualize connectomes in 3-dimensional geometries to see the algorithm's performance visually? (There are opensource libraries that could support this, e.g., geomstats)

Rating

6: marginally above the acceptance threshold

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good