Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus

Summary

This paper aims to improve LLM's logical reasoning ability by constructing synthetic data used in continual training. This work is largely built upon FLD, and proposes four other design principles for the synthetic dataset. Namely, reasoning with unknown facts, illogical reasoning, diverse reasoning rules, and linguistic expressions. By training llama-7b and 70b models on the synthetic dataset, the authors proved the effectiveness of their methods by achieving improvement on various benchmarks such as BBH.

Strengths

1. The experiments are pretty solid. The authors tested ALPT on different scales of LLMs on various NLP tasks such as logical reasoning, reading comprehension, etc. 2. According to Table 2, the performance gain on 70b model is even larger than 7b model, which shows the potential of ALPT. 3. The paper is organized well. 4. The authors provide an anonymous link for all the code, model, and data to ensure reproducibility.

Weaknesses

1. I am mostly concerned about the contribution of this paper. It seems to me that the four design principles are incremental improvements based on [1] by increasing the vocab size and enriching the diversity of logical rules and linguistic expressions. The results in Table 2 seem incremental compared with FLD baselines either. Also, the writing of some parts of the paper is very similar to [1], with even the same examples. 2. The current writing makes it hard for non-experts to understand. Specifically, the description of how the generator works is listed in the appendix. If one hasn't read [1] or the appendix, it would be very hard to understand the overall workflow. 3. It would strengthen the paper if the authors could also provide the results for FLD in Table 3 as a baseline for comparison. [1] Morishita, Terufumi, et al. "Learning deductive reasoning from synthetic corpus based on formal logic." International Conference on Machine Learning. ICML, 2023.

Questions

1. I think the training process using the constructed synthetic data is more like a continual training or SFT setting. Calling it additional "pre-training" could be a little misleading.

Rating

4

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

There is a limitation section in the appendix.

Summary

This work proposes Additional Logic Pre-Training (ALPT) to enhance logical reasoning abilities using synthetic rule-based data. The paper first discusses the design principles for creating a logical corpus and subsequently builds PureLogicDiverse (PLD). By training on PLD with RecAdam, models demonstrate improved performance in logical reasoning across various tasks and datasets, effectively integrating enhanced logical reasoning capabilities with their inherent knowledge. The results confirm the effectiveness of the proposed dataset and the ALPT strategy.

Strengths

1. The paper is well-written and presents a fluent narrative from the design principles to the creation of the dataset. 2. Compared to previous logical datasets, PureLogicDiverse contains more comprehensive and complex scenarios, serving as a useful resource for the community. 3. After additional logical pre-training on PLD, different models achieve better performance on both logical reasoning and NLI tasks, with no loss on other tasks. This training strategy together with the PLD dataset can be of interest for the development of future models. 4. The experiments are solid and sound. The comparisons and ablations are comprehensive, providing clear analyses of the effects of different design principles.

Weaknesses

The overall experiments are well-established. However, when comparing the ablation results with those of previous work (Table 2a and Figure 2), RuleTakers appears to perform well enough given the simplicity of its design. For example, comparing ‘ALPT-RT’(Table 2a) with ‘w/o DP2’(Figure 2), despite ‘w/o DP2’ containing much bigger vocabulary size and more extensive rules and expressions, these two settings gain similar performances. To clarify these comparisons, it would be helpful if the authors could provide statistics on the different dataset settings, such as the number of steps contained in the sampled training splits of each corpus and the average number of rules/steps per sample.

Questions

1. In dataset creation (Line 188), when sampling from the predefined logical formulas, is there a prior distribution from which to sample, or are the formulas evenly distributed? Do you have statistics on the operators and formulas of the created dataset? 2. Since the nouns and predicates are randomly composed to form logical expressions, they lack practical meanings. Intuitively, the coverage of rules and formulas plays a more important role than vocabulary coverage. Considering this, is covering a wide range of 15k vocabulary really necessary? As shown in Figure 2 (w/o DP1), restricting the vocabulary size appears to have minimal impact on the final performance and could also help decrease the dataset size, potentially improving data efficiency. Do you have any insights on this?

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

The authors have reasonably discussed the limitations and potential societal impacts of their research.

Summary

The paper discusses a novel approach to improve the logical reasoning capabilities of large language models (LLMs). The authors propose a method called Additional Logic Pre-Training (ALPT), which involves training LLMs on a synthetic corpus named PureLogicDiverse. This corpus is designed to include high-quality, program-generated reasoning samples that adhere to strict logical principles.

Strengths

The approach significantly enhance LLMs in logical reasoning abilities. Empirical results show that models pre-trained with ALPT on the PureLogicDiverse corpus perform much better, especially on benchmarks like BBH and NLI tasks. This demonstrates that ALPT can make LLMs more versatile and capable of handling various reasoning tasks. The systematic design of the PureLogicDiverse corpus ensures that the reasoning samples are high quality, covering a wide range of logical rules and linguistic expressions. This comprehensive method not only boosts logical reasoning but also helps models integrate existing knowledge with new reasoning skills. As a result, they become more effective in tasks requiring both logic and knowledge.

Weaknesses

There is a risk of models overfitting to the synthetic logic patterns, which could limit their generalizability to real-world applications. The success of ALPT heavily depends on the quality of the design principles used to create the synthetic corpus, and any deficiencies in these principles could reduce its effectiveness. Therefore, while ALPT shows great promise, it comes with challenges that need careful management to maximize its potential benefits.

Questions

Authors discussed about the performance on other tasks like CommonsenseQA and Hellaswag, the results show that there is no substantial improvement on these tasks. I am curious to see the performance on other tasks like math or coding.

Rating

8

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The approach may not fully address tasks requiring complex procedural understanding or multiple-choice questions that involve nuanced reasoning. Further research is needed to integrate ALPT with other methods to enhance performance on these types of tasks. he effectiveness of ALPT heavily depends on the quality and comprehensiveness of the design principles used to create the synthetic corpus. Any biases or limitations in these principles could affect the overall performance improvements and generalizability of the models.