Will the Real Linda Please Stand up...to Large Language Models? Examining the Representativeness Heuristic in LLMs

Summary

The authors create a new dataset, ReHeAT, that assesses susceptibility to the representativeness heuristic (RH), where humans judge scenarios similar to an exemplar to be more likely. The authors show that many popular LLMs exhibit RH biases, and that this bias can be improved by nudging the models towards using its in-weights knowledge. The paper is clear and polished, and the experiments are thoughtful and extensive.

Rating

8

Confidence

4

Ethics flag

1

Reasons to accept

- New benchmark for assessing language models for a known bias in humans. - Concrete takeaways for the reader to fix this bias in their models. - Thoughtful core premise supported by logical and interesting experiments.

Reasons to reject

I recommend acceptance.

Questions to authors

Comments: - Change Subtitle 5.3: "Improving Performance by Hinting (that LLMs to ?) use Their Knowledge" - Discussing your results in light of human base rates would be interesting. I see the note on page 8 that human base rates are around 10 to 50%. In a way, should we expect LLMs to exhibit this bias, in that they are trained to replicate the judgments of a population of humans?

Summary

This paper evaluates the impact of the representativeness heuristic on LLMs' reasoning. It proposes a dataset that covers various types of representativeness heuristic. Experiments show that LLMs make representativeness heuristic errors similar to humans and they behave differently when faced with different types of questions. In addition, providing hints in the prompts improves models' performance. The paper highlights the importance of considering the representativeness heuristic when using LMs for reasoning and decision-making.

Rating

8

Confidence

3

Ethics flag

1

Reasons to accept

- Representativeness heuristic on LLMs is less explored in prior work, but understanding its impact on LLMs’ reasoning is important. - This paper is well-written and easy to follow. The question types and experimental setup are clearly described. Examples of each type of questions are detailed in the appendix. - The experiments are well-designed, especially the controlled ones on statistical SPQs and IQs. The qualitative analysis is comprehensive and interesting.

Reasons to reject

- This paper aims to investigate whether LLMs make representativeness heuristic mistakes similar to humans, but understanding the results is challenging without human performance for comparison. It would be helpful if the authors could conduct similar human studies or, at the very least, provide references from previous studies, given that some questions are directly drawn from prior research. - The paper mentions “49 questions drawn directly from previous research and 153 new questions”, but the curation process for the new data is unclear. Also, how diverse are the new questions?

Questions to authors

- The experiments on IQs and SPQs appear to only use one example per question type, but it remains unclear whether this performance would remain consistent with more examples. It would be better if the authors could include results using more examples. - How are the ICL examples selected in section 5.1? - Missing reference that compares human and LLMs reasoning, e.g. [1] [1] Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan, Hannah R. Sheahan, Antonia Creswell, Dharshan Kumaran, James L. McClelland, Felix Hill. Language models show human-like content effects on reasoning tasks. ## Comments after Author Discussion Rating: 7 → 8

Summary

This paper investigates an interesting topic, i.e. whether LLMs show typical representativeness heuristic as in human decision-making process. The authors construct a test set covering six types of representativeness heuristics within the field of psychology, and evaluate multiple existing LLMs including GPT-3.5, GPT-4, PaLM 2 and LLaMA 2. Extensive experiments show that all LLMs demonstrated similar representativeness heuristic bias to humans, and advanced prompt techniques such as COT, self-consistency can only bring marginal improvements. Overall , I think this is a good paper. Studying and understanding what kind of shortcoming LLMs have inherited from texts produced by human is important to build intelligent systems that are smarter than human, and I believe this paper takes an important step towards this goal. The topic also perfectly fits the theme of COLM. The paper also shines in its strong experiments, discussing multiple prompting techniques to improve models' performance on the proposed REHEAT dataset.

Rating

8

Confidence

4

Ethics flag

1

Reasons to accept

- important research topic - thorough experimental results - a newly constructed datasets

Reasons to reject

- no obvious shortcomings

Questions to authors

- Do you think more sophistic prompting strategies such as multi-agent debating can bring larger improvements? - It would be better to provide experiments of testing models performance across different scales. - Is there any results on average human accuracy on the proposed testset?

Summary

This paper presents an evaluation suite testing language models for the "representativeness heuristic" cognitive bias. It reviews six types of this heuristic, with illustrations, detailed descriptions, and examples, and reviews the results of testing LMs for these biases. The paper includes results for: * Four LLMs, * four prompting approaches, * varying numbers of few-shot examples, * correctness of the models' reasoning, * two methods of 'hinting' the models, and * two easier sets of related questions ("SPQs" and "IQs") testing necessary knowledge for solving the problems. These experiments show that LMs exhibit qualitatively similar behavior to humans on such questions.

Rating

8

Confidence

3

Ethics flag

1

Reasons to accept

This is an interesting and well-executed study of LM reasoning. The paper is super well written and clear, focusing clearly on a question and testing it thoroughly. Lots of extra details are given in the appendix. This is probably a good evaluation to add to the long running list of interesting LM evaluations to run on frontier models. Lots of different settings are tried and tested and results are reported super clearly and legibly. I appreciate the text coloring and sizing in the tables. Overall just a nice paper.

Reasons to reject

Definitely not worth rejecting over, but I think what would really improve this paper a lot is spending some time discussing construct validity and external validity. Why are we testing models for these kinds of biases? What do we conclude about them when they get these results? With humans, results on these tests may be taken to be indicative of broader reasoning patterns outside of the testing setting. But we already know that can be pretty problematic — psychology tests are run on a select group of people in a very artificial environment, subjects generally try to perform the way they think they are expected to perform, and the results may not be representative of how people act in the real world (See [this paper](https://www2.psych.ubc.ca/~schaller/528Readings/Rozin2001.pdf) for a nice discussion of this in the context of social psych). So there are important caveats to these kinds of results, but we take them seriously because of what we understand about the mental processes behind human decision making. We do not have a comparable understanding of LM decision making, into which we do not have introspection, and which is much less consistent and systematic than human decision making. So what does it mean that LMs respond like humans do? Perhaps it just means they were trained on human cognitive bias tests, and any other place where they are presented with the opportunity to do such reasoning, they will behave very differently! Or perhaps these biases will show up in quite a different way when the LM is presented with a more complex or compound problem to solve, and its implicit reasoning won't follow the same patterns. It's really hard to say how these kinds of tests generalize to explain human behavior more broadly, and the case is much, much harder for LMs. Not to say it's not worth doing and trying, but I think this is an important scientific consideration and a serious limitation of this kind of study, and I think the paper would be much better if it addressed it carefully.

Questions to authors

* More of a comment: the base rate fallacy picture doesn't seem to be a great example of the fallacy. Would be nice if you could illustrate it there too. Unless I'm missing something? * The caption on the Disjunction Fallacy image is ungrammatical.