Library

LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models

Generating accurate step-by-step reasoning is essential for Large Language Models (LLMs) to address complex problems and enhance robustness and interpretability. Despite the flux of research on developing advanced reasoning approaches, systematically analyzing the diverse LLMs and reasoning strategies in generating reasoning chains remains a significant challenge. The difficulties stem from the lack of two key elements: (1) an automatic method for evaluating the generated reasoning chains on different tasks, and (2) a unified formalism and implementation of the diverse reasoning approaches for systematic comparison. This paper aims to close the gap: (1) We introduce AutoRace for fully automated reasoning chain evaluation. Existing metrics rely on expensive human annotations or pre-defined LLM prompts not adaptable to different tasks. In contrast, AutoRace automatically creates detailed evaluation criteria tailored for each task, and uses GPT-4 for accurate evaluation following the criteria. (2) We develop LLM Reasoners, a library for standardized modular implementation of existing and new reasoning algorithms, under a unified formulation of the search, reward, and world model components. With the new evaluation and library, (3) we conduct extensive study of different reasoning approaches (e.g., CoT, ToT, RAP). The analysis reveals interesting findings about different factors contributing to reasoning, including the reward-guidance, breadth-vs-depth in search, world model, and prompt formats, etc.

cs.CLcs.AINatural Language Processing TechniquesComputer Science
View graphSource

Paper

Similar papers

Structured Reasoning with Large Language ModelsSrihari Tanmay Karthik Tadala2025 · Preprints.org In Library
Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning PerformanceYao Fu, Litu Ou, Mingyu Chen et al.2023 · arXiv (Cornell University) · 134 citations In Library
A Survey of Optimization and Evaluation for Large Reasoning ModelsXiaofei Dong, Xi Zhang, Dan Zhang et al.2025 In Library
The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure ModesAvinash Anand, Mahisha Ramesh, Avni Mittal et al.2026 · 0 citations In Library
Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMZhen Xiong, Yujun Cai, Zhecheng Li et al.2025 · arXiv (Cornell University) · 19 citations In Library
Enhancing Reasoning in LLMs through Contrastive Estimation and Representation EngineeringTim Condello2025 In Library
Advancing Reasoning in Large Language Models: Promising Methods and ApproachesAvinash Patil2025 · arXiv.org · 60 citations In Library
Towards Reliable Reasoning in Large Language Models: A Systematic Review of Chain-of-Thought and BeyondANURAG KUMAR SINGH2026 · Zenodo (CERN European Organization for Nuclear Research) In Library
Towards Reliable Reasoning in Large Language Models: A Systematic Review of Chain-of-Thought and BeyondANURAG KUMAR SINGH2026 · Zenodo (CERN European Organization for Nuclear Research) In Library
Advancing Large Language Model Reasoning Techniques: Methods Enabling LLMs to 'Think' Beyond Text Generation for Reliable and Explainable AICassel Scott-Emuakpor2025 In Library

Peer review

Acceptcolmweb.org 2024 · via OpenReview
Reviewer bv3g6/10 · confidence 3/52024-05-08

Summary

This paper presents a new evaluation package of LLMs' complex reasoning capabilities. This paper has several contributions: (1) A new automatic evaluation metric, which first asks GPT4 to conclude common failure reasons of wrong responses on the trainined set and then ask GPT4 to evaulate models' output based on these reasons to achieve automatic evaluation. (2) A configurabale toolkit that allows user to specify their foundation model and reasoning strategy. In the end, this paper also presents a comprehensive evaluation of the peformance of llama2-70B with several reasoning strategies (i.e., CoT, RAP, and ToT)

Rating

6

Confidence

3

Ethics flag

1

Reasons to accept

1. This paper proposes a new evaluation system for LLMs' reasoning capabilities. 2. The comprehensive evaluation reveals certain limitations of existing LLMs.

Reasons to reject

The proposed method is hard to generalize. Based on the description in the paper, the proposed solution's generalization capability remains unclear. For example, the current system requires a set of questions (i.e., training data) to sample failure response for each task and then conclude the criteria list. As a result, this list might be only applicable to the current dataset. It would be great if some experiments can support the genralization capabilities of the criteria list. Regarding the unified library, it is unclear how to support reasoning search algorithm that requires the feedback from RMs since these models often require additional task-specific training.

Questions to authors

1. In the fourth paragraph of introduction, the paper mentrions that AutoRace and detect 70% of the incorrect reasoning chains. Can you elaborate more on that? This seems to be a moderate agreement. 2. Can you give a more clear definition of the mentioned world model? like what kind of world model can be supported by the current syste. I found it missing the first time this paper discussing the world model. 3. Regarding the criteria list, is it generalizable to other tasks? 4. Regarding the error collection, how to guarantee the coverage of the collected errors? Better to add an experiment to support this since this is the foundation of AutoRace. 5. In table 5, AutoRace is annotated as fully auto. However, if my understanding is correct, it still requires the support of a set of examples to construct the criteria list. Can you elaborate more on that?

Reviewer SuhS7/10 · confidence 4/52024-05-11

Summary

This paper presents principled methods on how to organize automatic evaluation of reasoning chains (AutoRace) and design a tailored evaluation library with a unified view of LLM reasoning algorithms. The authors conduct an extensive study on various configurations, including the choice of LLMs and promptings. Their experiments reveal some interesting findings, e.g., search breadth is superior to depth in LLM reasoning.

Rating

7

Confidence

4

Ethics flag

1

Reasons to accept

1. The proposed AutoRace is fully automatic and can achieve comparable performance to baselines with expert-crafted annotations. 2. The unified formulation of LLM reasoning is a good principle for implementing evaluation systems and allows for a systematic comparison across reasoning algorithms. 3. Extensive experiments suggest intriguing observations and provide best practices and insight for LLM reasoning. These experiments also validate the efficacy of their system.

Reasons to reject

1. Novelty could be somewhat limited, but the system development is intensive and the software could be useful for the community. 2. AutoRace relies heavily on the availability of training sets. However, constructing such a training set for a specific reasoning task is expensive and contradicts the main purpose of AutoRace, which aims to reduce human efforts. This weakens the claim of this paper.

Reviewer rws27/10 · confidence 4/52024-05-18

Summary

The paper introduces AutoRace for automated evaluation of LLM reasoning chains and LLM Reasoners, a library standardizing reasoning algorithms for systematic comparison. These tools enable detailed analysis of reasoning methods, revealing insights into factors like reward guidance, search strategies, world models, and prompt formats.

Rating

7

Confidence

4

Ethics flag

1

Reasons to accept

- The work presents an interesting framework for evaluating reasoning tasks. - They nicely formulated the reasoning problem under a common framework.

Reasons to reject

- The steps taken in Autorace to annotate are not clear. - The empirical findings are mostly known. However, after the rebuttal phase, I can see enough novelties in the empirical findings. - No comparison in Autorace annotation quality.

Questions to authors

-Can you give an estimate of how accurate auto race annotations are?

Reviewer 5tZW6/10 · confidence 4/52024-05-21

Summary

This paper proposes a framework (AutoRace) to automate the evaluation of large language models (LLMs) on reasoning tasks. Specifically, the framework evaluates the reasoning chains generated by LLMs. The authors have created a library to formalize the evaluation of existing reasoning tasks across various prompting techniques, such as chain-of-thought, beam search, tree-of-thought, and reasoning-via-planning.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

1. The paper focuses on automating the reasoning chain evaluation task, which can be a key to standardizing LLM evaluation. 2. The authors have provided a comprehensive comparison with contemporary techniques and demonstrated the superior performance of AutoRace.

Reasons to reject

1. The authors described their evaluation algorithm but did not provide the pseudocode. 2. In the last paragraph of section 3.1, it is unclear whether the criteria list is predefined or generated by GPT-4. If GPT-4 generates the list, how is its consistency maintained? 3. It is unclear which reasoning chains the evaluator can correctly assess and which ones it misses. An ablation study is needed. 4. Are there any biases introduced by the LLM evaluator?

Reviewer SuhS2024-06-02

Re: Rebuttal by Authors

Thanks for your response! The additional experiment addresses my concern about the sample efficiency of AutoRace. However, I noticed an interesting phenomenon in the new result, where providing more incorrect reasoning chains to GPT-4 does not yield a consistent gain (at least for 2/6 datasets). Do you have any explanation for that?

Authorsrebuttal2024-06-04

Re: Re: Rebuttal by Authors

Thanks for your reply! One possible reason is that the criteria list constructed by GPT-4 given more examples includes some general criteria, such as `relevant to the question` . These criteria may not be specific enough to help improve evaluation. How to make use of more examples to consistently improve evaluation accuracy remains an interesting problem, and we plan to explore it in future work. For example, it's possible to use a small validation set to select the best composition of criteria, or even find the best criteria list with some tree search algorithms [1]. [1] Wang et al., 2023, "PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization"

Reviewer bv3g2024-06-02

Thanks for the reply. I have increased the overall rating from 5 to 6. Your results on GSM8K are encouraging. However, I still think designing critics based on a dataset is not the most convincing way because we cannot know what type the user query belongs to in a real system. I know it is hard, but trying to think out of specific tasks and find more fundamental critics (or letting the model identify by itself) could make the system more helpful in real applications.

Authorsrebuttal2024-06-04

Re: Official Comment by Reviewer bv3g

Thank you for your feedback! Our current work focuses on analyzing LLM reasoning, and constructing an automatic reasoning chain evaluation metric, even though tailored to several reasoning datasets, takes a concrete step towards this objective (e.g., it helps reveal the novel conclusions in Section 5). We recognize the importance of developing an evaluation or reward model for general purposes. Such a model remains a goal for our long-term research efforts, and we are dedicated to advancing towards it. For example, a natural next step of AutoRace would be using human feedback to learn a better criteria list that applies to general real-world applications.

Authorsrebuttal2024-06-05

Looking forward to the discussion

Dear reviewer, We have dedicated ourselves to thoroughly addressing all your concerns and questions in the rebuttal. Could you please spare a moment to review our response? We are eager for your feedback and prepared to provide any further clarifications. Thank you for your time and consideration.

Authorsrebuttal2024-06-05

Looking forward to discussion

Dear reviewer, We sincerely appreciate your insightful feedback once again. In response, we have diligently addressed each of your concerns and queries. Notably, we've supplemented the pseudo-code and multiple failure cases of AutoRace. Would you be able to spare some time to review our response and confirm if it adequately addresses your questions? We are eager to engage in further discussions and provide additional clarifications on any new queries you may have. Thank you for your time and consideration.

Reviewer rws22024-06-05

Thanks for clarifying my comments. Most of my questions are clarified with your detailed response. Can you shed some light on how accurate auto-race annotations are?

Authorsrebuttal2024-06-05

Thank you for your feedback! The average accuracy of AutoRace is 0.86 (from the experiment results in Table 1). This is higher than the best baseline, which achieves an accuracy of 0.82. Moreover, AutoRace is the only method that performs within 5% of the best results across all six datasets, showing outstanding robustness. For more experimental results and detailed analysis, please refer to Section 3.2 in the paper. We are happy to answer any further questions you may have.

Reviewer rws22024-06-05

Thanks for the prompt reply. Let me clarify the question. In Table 1, AutoRace performs better than human reasoning in some instances. That is why I was asking how accurate the basis of judgment is. Can you explain why AutoRace can be better than a human evaluator in some instances?

Authorsrebuttal2024-06-06

In Table 1, we didn't compare AutoRace to human evaluation. In fact, we use human evaluations of reasoning chains as the **ground truth labels** (True/False) in the experiment. Instead, we include **`answer-based evaluation`** as a reference in Table 1. --- "Why can AutoRace be better than **answer-based** evaluation in some instances?" Answer-based evaluation evaluates reasoning chains solely based on whether the final answer matches the ground truth answer, disregarding the correctness of the intermediate reasoning steps. This can lead to the **false positive** problem (as demonstrated in Figure 1, a), where a reasoning chain might conclude with a correct answer but be supported by faulty reasoning. In contrast, AutoRace is specifically designed to assess the reasoning process by applying learned evaluative criteria.

Reviewer 5tZW2024-06-07

Thank you for your detailed rebuttal. You have addressed most of my concerns. However, I would like to make two additional observations: 1. The comments in the pseudocode for steps 3 and 4 appear to be swapped. 2. Human study evaluations are necessary to establish the reliability and soundness of the criterion lists.

Authorsrebuttal2024-06-07

Thanks for your reply. **comments in the pseudo-code** We have fixed the typo and updated the pseudo-code [here](https://anonymous.4open.science/r/rebuttal-F471/autorace-algo-fix.pdf). **human evaluation of criterion lists** Our experiments demonstrate a strong alignment between AutoRace and human evaluations of reasoning chains. The overall performance of AutoRace serves as an effective indicator of the quality of criterion lists. We have also uploaded all criteria lists ([link](https://anonymous.4open.science/r/rebuttal-F471/autorace-criteria.txt)) used in our experiments. The authors have manually examined them, and concluded that all of them cover diverse dimensions of evaluation and contain no obvious mistakes. Thanks again for your feedback. We will add the updated pseudo-code and all criterion lists into the paper.

Program Chairsdecision2024-07-10

Decision

Accept

© 2026 NYSGPT2525 LLC