ARB: Advanced Reasoning Benchmark for Large Language Models

Thank you for the review; here are the answers to your questions. > As the limitation states, this benchmark could have potentially be seen through the training dataset and lead to the contamination problem. Thank you for pointing this out. We do not have access to the training set of any of the tested models. In spite of this, we can test for memorization in this manner: we ask models to complete problem statements randomly sampled from the dataset, and compare their outputs against the true continuation of the problem statement. (Please see Appendix G for details.) As the similarity scores are quite low, we claim that models we test do not easily complete the exact problem statements, which rules out the naive memorization hypothesis. However, memorization in LLMs is a complex topic, and the fact that the text of a problem is not memorized does not necessarily mean the model has not memorized a similar problem or fact; we also discuss this in Appendix G. We are not aware of other works that successfully resolve this issue. > Among the problems, how many of the symbolic problems can be convert into concrete calculation based problems? Thank you for asking this. The answer to this question strongly depends on what is meant by “concrete calculation,”. However, if we were to define this to mean that the ground truth solution mostly consists of equations, then around a third of the math problems satisfy this criteria. As for the physics problems, almost all of the problems are concrete calculations (almost by nature of the subject).

We are grateful for the positive review; here are the answers to your questions. > It’s not intuitive what questions are covered by ARB. Can you put a few examples in the paper? Thank you for the suggestion! We do provide examples along with model outputs in Tables 10 through 13 (numbering of the rebuttal version). We now include some more examples in Tables 7 through 9 of the Appendix. > It’s not clear what the position of ARB is compared to existing benchmarks. The authors claim ARB is more difficult. In our updated version we do a better job of placing our work in context of the very recent literature. The most relevant seems to be [1], which is a similar work as ours, developed concurrently. The main differences are: (1) their quantitative problems are slightly easier than in our benchmark, and require much less deep math knowledge; (2) our math/physics questions are not multiple-choice; (3) their quantitative problems are drawn from a single source which are somewhat more likely to be thoroughly discussed online. We also cite, among others, [2], which is indeed quite challenging, but is in a different area. We moderated our language (in the paper and in the abstract) to avoid claiming that our benchmark is uniquely challenging for current models. > Writing errors Thank you, we have now fixed the “aspirational” sentence and the units in the figure. [1] Arora, D., & Singh, H. G. (2023). Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models. arXiv preprint arXiv:2305.15074. [2] Valmeekam, K., Olmo, A., Sreedharan, S., & Kambhampati, S. (2022). Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change). arXiv preprint arXiv:2206.10498.

Thank you for the review and the questions! We answer your concerns one by one. > The solutions to these problems sets from textbooks may already in the training data. How to deal with this situation? Unfortunately, we do not have access to the training set for any of the tested models. In spite of this, we can test for memorization in this manner: we ask models to complete problem statements randomly sampled from the dataset, and compare their outputs against the true continuation of the problem statement. (See Appendix G for details.) As the similarity scores are quite low, we claim that models we test do not easily complete the exact problem statements, which rules out the naive memorization hypothesis. However, memorization in LLMs is a complex topic, and the fact that the text of a problem is not memorized does not necessarily mean the model has not memorized a similar problem or fact; we also discuss this in Appendix G. We are not aware of other works that successfully resolve this issue. > How do you grade the types of mistakes GPT-4 make in Table 3? Is it human evaluation or are there some hard-coded rules. It is human evaluation. We say in Section 4.2: "It is possible that our graders underestimate the rate of arithmetic mistakes in some cases, especially when the approach is clearly wrong, or it is not clear whether a given error is due to faulty reasoning or due to a missed term in the calculations." The graders were given the descriptions of the five general types of errors as guidance; in addition, the graders were familiar with the relevant section of [1]. We are not sure automated evaluation of detailed error types is possible using current technology; the only way to reduce noise would be to employ more graders. > It is also unclear how the current low performance on the ARB benchmark is not due to under-claiming. Thank you for bringing this up. We definitely agree that there is no way to prove that models cannot do this, as our prompts are surely not optimal. It is not clear to us, at this stage, what the “best” way to prompt and use models for our benchmark; and we think finding out the answer is outside the scope of this benchmark paper. We now additionally do one-shot testing on the Numerical part of the benchmark to see if anything improves, and it doesn’t. [1] Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv preprint arXiv:2303.12712.

Thank you for the very useful review. Here are the responses to your concerns. > I view this work as an additional benchmark to the existing ones that involves a broader set of domains. Thank you for emphasizing those lines of work! We do cite [1], which is a similar work as ours, developed concurrently. The main differences are: (1) their quantitative problems are slightly easier than in our benchmark, and require much less deep math knowledge; (2) our math/physics questions are not multiple-choice; (3) their quantitative problems are drawn from a single source which is somewhat more likely to be thoroughly discussed online. The updated (Oct 2023) version of [1] also discusses a version of self-evaluation in the “Can GPT-4 find and correct its mistakes” section, and find a negative result using some simple prompts. We introduce rubric-based self-evaluation as a way to make progress on this. We also cite [3] (the followup to the mentioned [2]), which is indeed quite challenging on a second glance, but is in a different area. We moderated our language (in the paper and in the abstract) to avoid claiming that our benchmark is uniquely challenging for current models. > Moreover, it’s not entirely clear how solving the benchmark (especially MCAT or Law questions) would correlate with reasoning ability. We agree that the term “reasoning ability” does not have a clear definition in the literature. Our estimate is that some of the law problems are quite nontrivial for a non-trained human, and the sources agree. It is possible that the main abilities required to solve it are reading comprehension and pattern matching from memorizing a lot of legal cases, and not “reasoning” as in the math or physics problems. > Is a non-parsable answer deemed as an incorrect answer? Or is that question not even considered in the final evaluation? Thank you for bringing this point up. We do discuss this in the Evaluation section. For numerical problems, we prompt the model to respond with the final answer, and if the final expression is not parseable using Python’s SymPy library, it is marked as incorrect. (For physics problems, there is the additional step of specifying what units to use for the final answer.) The process is harder for symbolic answers since they have less structure and are generally harder to parse. We use SymPy to check equivalence between model output and the ground truth up to a permutation of variables. If the two are not equivalent or the model output is unparseable, then the answer is marked incorrect. Manual inspection revealed little discrepancy in the scoring compared to an "ideal" equivalence checker. > Diagonalization procedure. Given the rapid iterations of LLMs with new and more extensive training data, a diagonalization procedure could be worth investigating, We do agree that novel methods that maintain utility of all recent benchmarks after publication would be great; however, this was somewhat outside the scope of this project. We try to keep our data behind an API, accessible by requests that are not explicit links, so simple crawlers will not just ingest it; and “smart” LLM-based data collection mechanisms hopefully realize it’s a benchmark. In the current paradigm, it is impossible to prevent overfitting in evaluation if model creators are not careful. [1] Arora, D., & Singh, H. G. (2023). Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models. arXiv preprint arXiv:2305.15074. [2] Valmeekam, K., Olmo, A., Sreedharan, S., & Kambhampati, S. (2022). Large Language Models Still Can't Plan (A Benchmark for LLMs on Planning and Reasoning about Change). arXiv preprint arXiv:2206.10498. [3] Valmeekam, K., Marquez, M., Sreedharan, S., & Kambhampati, S. (2023). On the Planning Abilities of Large Language Models: A Critical Investigation. arXiv preprint arXiv:2305.15771.