Exploring the Mystery of Influential Data for Mathematical Reasoning

Summary

This paper mainly studies data selection for supervised instruction tuning of LLMs, targeted at math reasoning. Specifically, it follows existing data selection work that selects supervised tuning data based on the combination of its diversity and quality. Experiments on large-scale open-source data show that the proposed data selection approaches outperform (not by a big margin) both random selection and other baseline data selection approaches.

Rating

4

Confidence

4

Ethics flag

1

Reasons to accept

1. The experiment on math reasoning is solid, as it evaluates the proposed data selection from multiple perspectives. 2. The paper is relatively easy to understand.

Reasons to reject

1. My primary concern is the motivation behind the paper, specifically the research question addressing the math reasoning task. While the paper focuses on math reasoning, the proposed method appears universally applicable to various tasks. After a thorough review, I found no clear motivation or adaptation of the method specifically for the challenges in math reasoning. An interesting idea might involve a compositional aspect, such as whether different combinations of formulas could lead to improvements. 2. The reported improvements in results are quite marginal. It is unclear what a ~1.5-2% improvement in math reasoning signifies in this context. Such modest gains might be overshadowed by the capabilities of a base model, such as LLAMA 70B. Consequently, the effectiveness of the proposed approach remains ambiguous.

Summary

This paper focuses on selecting influential training data for improving LLMs' mathematical reasoning ability. The authors propose a quality-aware diverse selection strategy (QaDS) over other selection strategies and also define an OpenMathMix which is a data mixture selected by QaDS. The experiments show advantages of the proposed strategy and the paper is comprehensive.

Rating

7

Confidence

4

Ethics flag

1

Reasons to accept

* Proposed a reasonable way of selecting high-quality data (one-shot better than zero-shot) * Define an influential data mixture with open-source data selected by QaDS

Reasons to reject

* Lack of further analysis of the selection strategy, e.g., more than one-shot

Questions to authors

Could this strategy be applied to other tasks than the mathematical reasoning?

Summary

__Summary__ 1. Propose a quality-aware diverse selection approach for mathematical reasoning. This approach aims to integrate both quality and diversity in selecting influential data. 2. Diversity: sampling by maximizing the minimum distance between the data points, which are encoding by embeddings. 3. Quality: the average times that one-shot performance is better than the zero-shot performamce. 1. They further train a quality scorer for that to reduce the intractable computational cost. 4. Then they can select the instance by the QS score. 5. Construct two settings $S_{base}$ and $S_{large}$ and different selection scheme to verify the effectiveness of QaDS, try to better demonstrate the effectiveness of QaDS

Rating

7

Confidence

4

Ethics flag

1

Reasons to accept

1. Motivation is clear and reasonable to incorporate both diversity and quality. The underlying approach is also solid. 2. Experiments demonstrate improvements with QaDS selection, also better performance compared to DeepSeek, which is the base model.

Reasons to reject

1. In Table 4, althouh QaDS-$S_{large}-4.7M$ achieves the best performance $\mathbf{48.8}$, but according to Table 2, this OpenMathMix set contains all data for mathematical reasoning, only the "General" is selected by QaDS. Thus, it is confusing that the motivation is to improve mathematical reasoning, but we actually use all the corpus there. 1. And Table 4 only shows the performance of one dataset. 2. Although QaDS-$S_{large}-5M$ is worse, but again, we only perform QaDS on "General" according to Table 2, readers will be concerning about what's really improving here.

Questions to authors

Refer to "Reasons to Reject".

Summary

This paper aims to propose the data curation method which identifies highly influential data within a large-scale mathematical reasoning dataset. ### Quality The paper is generally clear and effectively communicates the problem it addresses. However, it lacks a crucial discussion on the computational costs involved in curating the dataset using the proposed method. ### Clarity The paper is well-written, but some sections may be challenging to grasp on first reading. ### Originality The application of the K-center Greedy method is not particularly novel, as it adapts an existing approach to a new problem. Nonetheless, the introduction of a quality scoring system using a zero-shot and one-shot inference appears to be a novel contribution. ### Significance The approach to curating the training dataset is significant. This paper addresses the important issue of fine-tuning language models for reasoning datasets.

Rating

6

Confidence

3

Ethics flag

1

Reasons to accept

- **Thorough analysis:** This paper introduces a strategy for selecting data for a mathematical reasoning dataset. It applies this method to two different dataset sizes, S_{base} and S_{large}, demonstrating that the proposed method outperforms existing baselines. Additionally, it provides insight into the relationship between estimated and actual quality scores, showing a strong correlation with the actual scores.

Reasons to reject

- **Needs clearer writing**: Some parts of the paper are difficult to follow. For example, it is not clear which dataset is referred to in Table 3. The explanation of the scoring method in Figure 1, particularly the bi-directional arrows between the one-shot and zero-shot score blocks, is also confusing. - **Insufficient discussion on computational costs**: The paper does not adequately address the computational costs required to embed data for diversity measurement and computing zero-shot and one-shot scores using LLMs like LLaMA-2 and Mistral. The cost implications of these computational demands, especially when seeking a modest accuracy improvement of 2% as shown in Table 4, need further discussion.

Response to authors

> General data can also be influential for reasoning: According to [1] and [2], other relevant tasks can also be helpful for reasoning. So we contain general data in our setting, and successfully selects the helpful general data by QaDS. But again, am I right that, QaDS-S_{large}-4.3M (48.8%)> QaDS-S_{large}-3M (45.2% ) (Table 4), here the only difference is that if the math data is selected by QaDS. But turns out using QaDS in this case is worse. AND, in the motivation, the goal seems to select data for mathematical reasoning. (Abstract below) > To go further, there exist two open questions for mathematical reasoning: how to select influential data and what is an influential data composition. For the former one, we propose a Quality-aware Diverse Selection (QaDS) strategy adaptable for mathematical reasoning. Thus, I also expect QaDS to select influential data in math data, rather than we simply use all of them. 2. Thanks for the experiments on other datasets.

Call for Discussion

Dear reviewer gRx8, Thank you very much for taking the time to review our paper and provide valuable feedback. As the discussion period is coming to an end, we would like to ensure that our recent response satisfactorily addresses your concerns. If you have any further concerns, you are welcome to point them out, and we are happy to clarify them further. Thank you! Best， Submission310 Authors