Reasoning about concepts with LLMs: Inconsistencies abound

Summary

This paper seeks to address a well-known issue in LLMs: inconsistency in knowledge. The strategy is to use ontologies to extract Is-A relations and then study them in LLMs, with a focus on applicability to real-world tasks. They find that models have a lot of knowledge, but that they are inconsistent in this knowledge. They show improvements by using KB-based prompting. This paper addresses an important topic, is clearly written, and has a promising and interesting approach. I would be happy to see it published with some revisions, but I have a few comments and suggestions. First, a strength is that the description of ontologies (transitivity, etc) is very clear and accessible. This would I think be valuable for the audience. But, perhaps as a trade-off, I found Section 6 very rushed, with most of the important details about the key prompting method in the Appendix. I would recommend putting more detail in the main text, perhaps with an illustrative figure for the prompting method. Some of the relevant detail is in Fig 1, but it would be good to focus on exactly the prompting part. Second, I was somewhat surprised by the discussion of not counting cases where the LLM gives mealymouthed answers like “it is possible that an orthopedic pediatric surgeon may work in the field of infection control, however this is not their primary field of occupation, which is orthopedic surgery and pediatric surgery”. This is increasingly a strategy imposed on LLMs. It’s perhaps fair (although I’m not sure) to count these as inconsistent vis a vis yes/no. But, if a lot of the apparent inconsistency comes from such examples, it would be good to know. The scenario where an LLM hedges like that in one case and simply says “no” in another feels very distinct from a yes/no asymmetry. I would address this explicitly in the main text. A ubiquitous question: I was left wondering what, if anything, should we take away from the differences in model performance. It’s nice to test a lot of models, but are there hypotheses as to if/why some models might be better than others on this particular task.

Rating

8

Confidence

3

Ethics flag

1

Reasons to accept

- This tackles an important topic in a clear way: making LLMs more trustworthy and consistent in the way that KBs are while retaining strengths of LMs. - It’s clearly written and might help introduce some of the ontology-based thinking to the LLM audience in a helpful way. I found the literature reviews useful for understanding the place in the literature. - The method seems to work and show meaningful improvements, and it seems scalable.

Reasons to reject

- Not really a reason to reject, but I’d recommend more detail on the prompting method. - The counting of hedged LLM answers as inconsistent makes me wonder if the inconsistency is overstated.

Questions to authors

I take it cafe racer and naked bike are real examples? I wasn’t familiar with them and so was unsure if they were supposed to be “novel concepts”. I was briefly thrown by the term “field of occupation”, which doesn’t seem idiomatic to me.

Summary

This paper devised methods that utilize ontologies to assess the consistency and coverage of conceptualization in Large Language Models (LLMs). By automatically generating test cases from knowledge graphs or ontologies, they effectively evaluate the models. Moreover, they have found that many prominent LLMs with publicly available weights exhibit inconsistencies in their knowledge, even when confronted with fundamental, small ontologies. However, through the application of straightforward prompting strategies, they have shown that it is possible to reduce these inconsistencies and improve the coverage of domain concepts across various LLMs with publicly accessible weights. Strengths: 1.A more detailed analysis of the hierarchical confusion across multiple large language models has been conducted; 2.The paper is well-crafted, featuring lucid clarity and a straightforward logical flow that is effortless to follow. Weaknesses: 1.The author focuses solely on hierarchical confusion while lacking necessary discussions on more comprehensive issues such as mixing of domains, inaccurate Generalizations, and contradictory information. 2.In terms of details, the analysis of the results in Table 2 and Table 3 is relatively rough. It just lists the results of different large language models on the issue without carefully analyzing the possible causes and influencing factors.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

1.A more detailed analysis of the hierarchical confusion across multiple large language models has been conducted; 2.The paper is well-crafted, featuring lucid clarity and a straightforward logical flow that is effortless to follow.

Reasons to reject

1.The author focuses solely on hierarchical confusion while lacking necessary discussions on more comprehensive issues such as mixing of domains, inaccurate Generalizations, and contradictory information. 2.In terms of details, the analysis of the results in Table 2 and Table 3 is relatively rough. It just lists the results of different large language models on the issue without carefully analyzing the possible causes and influencing factors.

Questions to authors

Is there a connection between the inconsistency of conceptual hierarchy and the parameter size of large language models? In other words, the larger the model parameter size, the lower the possibility of inconsistency in the conceptual hierarchy will be significantly reduced?

Summary

This paper studies the capability of LLM to reason with conceptual knowledge. It aims at testing LLM performance by evaluating conceptual consistency and the ability of handling logical inconsistencies. It proposes enhancing LLMs through knowledge integration.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

- The paper is well written and illustrated by figures. - The experiments and the use case show the potential of the proposed approach.

Reasons to reject

- Missing formal definition of inconsistency. - Missing comparison with approaches of ontology completion (instead of adding plausible rules, removing plausible inconsistency)

Questions to authors

Would it be possible to consider reducing inconsistency and improving the coverage of domain concepts as NLI tasks and fine-tune an LLM accordingly?

Summary

This paper presents an approach that leveraging ontologies and knowledge graphs to evaluate the conceptual knowledge understanding capabilities in several publicly available LLMs. The paper also presents simple prompting strategies to improve the domain concept knowledge in LLMs being evaluated.

Rating

7

Confidence

4

Ethics flag

1

Reasons to accept

This paper presents an interesting approach to assess the consistency and coverage of conceptualization in LLMs by automatically creating test cases based leveraging ontologies and knowledge graphs. Using these automated tests, the authors are demonstrate that several publicly available LLMs show inconsistent domain knowledge. The authors apply simple prompting approaches which allow the LLMs to respond with better understanding of certain medical domain concepts. The paper is well written and easy to understand. The experimental design and evaluation setting is decently thorough.

Reasons to reject

The paper presents an approach to evaluate several LLMs (lack of) understanding of medical domain concepts using simple IS-A hierarchical knowledge. However, domain understanding requires us to deal other more complex relationships including part-of, cause/effect, inference, entailment, etc. This paper would be better accepted by the conference audience if such evaluations were presented in the paper.