Thanks for your detailed, patient and insightful discussion. We respond to these questions as follows.
> **Q1**. Regarding W1, my concern about the limited applications remains. I acknowledge that K-hop sampling is a widely adopted approach and may not lead to performance degradation in some cases. However, for datasets like Cora and CiteSeer, the reported accuracy is much lower compared to pure GNN methods (Table 3). I am uncertain whether this is due to the k-hop sampling or because GITA only provides graph structures without node/edge attributes. If it is the latter, GITA may not be effective enough for handling graphs with attributes, whether in numerical or text format.
We think the reported inferior performance in Cora and Citeseer to the leading models is mostly because we do not use node attributes, but not due to the k-hop sampling. That is because many methods (e.g., Graph Transformer like UGT, and most GNN variants where $k$ is equivalent to the number of GNN layers) in your listed leaderboard also use k-hop sampling to achieve leading performance.
However, we want to point out that though using extra information beyond graph structures like node attributes is helpful (in Cora and Citeseer, they provide extra 0-1 binary word occurrences info), it requires specific models to handle them, thus hurts the generalizability, conflicting with our motivation, i.e., a "general-purpose" graph reasoning framework. Note that "generality" is a significant reason for why research works like GraphLLM and our GITA are interested in using LLMs for graph reasoning, where GraphLLM also does not consider the node attributes.
The hurt for the node attributes to generalizability can be reflected in several aspects: First, handling node attributes typically requires designing specific model tailored to their shapes (e.g., vector dimensions, length, matrix size), which hinders a general and consistent solution for handling diverse tasks. Second, the meanings (e.g., word vectors of titles, embeddings of degree or index, one-hot representation of node classes) of node attributes vary in data, thus the model can overfit in a specific task and hurt the zero-shot abilities. Finally, these diverse models to handle the node attributes also increase the complexity of the model.
As the major concern of GITA is how well vision+LLM performs on general graph scenarios, where node attributes are not necessary provided by default, we only concentrate on pure graph structures and do not incorporate task-specific models for handling extra node attributes.
Moreover, we can provide several potential solutions for combining GITA with node attributes, and leave them for future work.
1. We can include the text node attributes with the explanation of their concrete meanings inside the text prompt. However, such an approach may not perform well for some types of attributes such as occurrence data, because they are too abstract.
2. Similar to many existing works, we can use a specific module to encode the attributes and another fusion module to combine them with the backbone model (i.e., MLLMs used in GITA). However, such an approach needs extra designs for these additional modules and needs task-specific adjustments.
In the following table, We also provide an additional comparison between GITA and dedicated GNNs under non-attribute setting on Cora and Citeseer, where non-attribute setting is commonly used to evaluate how well models can understand graph structure. The experimental results show that GITA is much more effective in non-attributed Cora and Citeseer than dedicated GNNs, showcasing a more powerful awareness of their pure graph structure.
| Model | Cora | Citeseer |
|-------|-------|----------|
| GITA | 85.24 | 75.07 |
| GCN | 73.35 | 68.71 |
| SAGE | 69.19 | 64.69 |