Library

SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning

The pre-trained Large Language Models (LLMs) can be adapted for many downstream tasks and tailored to align with human preferences through fine-tuning. Recent studies have discovered that LLMs can achieve desirable performance with only a small amount of high-quality data, suggesting that a large amount of the data in these extensive datasets is redundant or even harmful. Identifying high-quality data from vast datasets to curate small yet effective datasets has emerged as a critical challenge. In this paper, we introduce SHED, an automated dataset refinement framework based on Shapley value for instruction fine-tuning. SHED eliminates the need for human intervention or the use of commercial LLMs. Moreover, the datasets curated through SHED exhibit transferability, indicating they can be reused across different LLMs with consistently high performance. We conduct extensive experiments to evaluate the datasets curated by SHED. The results demonstrate SHED's superiority over state-of-the-art methods across various tasks and LLMs; notably, datasets comprising only 10% of the original data selected by SHED achieve performance comparable to or surpassing that of the full datasets.

cs.CLcs.LGComputer Science
Source

Paper

References (59)

01FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank AdaptationsZiyao Wang, Zheyu Shen, Yexiao He et al.2024 · Neural Information Processing Systems · 188 citations In Library
02L-AutoDA: Large Language Models for Automatically Evolving Decision-based Adversarial AttacksPing Guo, Fei Liu, Xi Lin et al.2024 · GECCO Companion · 12 citations In Library
03What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction TuningWei Liu, Weihao Zeng, Keqing He et al.2023 · International Conference on Learning Representations · 411 citations In Library
04One Shot Learning as Instruction Data Prospector for Large Language ModelsYunshui Li, Binyuan Hui, Xiaobo Xia et al.2023 · Annual Meeting of the Association for Computational Linguistics · 55 citations In Library
05On Diversified Preferences of Large Language Model AlignmentDun Zeng, Yong Dai, Pengyu Cheng et al.2023 · Conference on Empirical Methods in Natural Language Processing · 30 citations In Library
06Rethinking the Instruction Quality: LIFT is What You NeedYang Xu, Yongqiang Yao, Yufan Huang et al.2023 · arXiv.org · 45 citations In Library
07LLM360: Towards Fully Transparent Open-Source LLMsZhengzhong Liu, Aurick Qiao, Willie Neiswanger et al.2023 · arXiv.org · 118 citations In Library
08MoDS: Model-oriented Data Selection for Instruction TuningQianlong Du, Chengqing Zong, Jiajun Zhang2023 · arXiv.org · 133 citations In Library
09Self-Evolved Diverse Data Sampling for Efficient Instruction TuningShengguang Wu, Keming Lu, Benfeng Xu et al.2023 · arXiv.org · 59 citations In Library
10Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive TasksPo-Nien Kung, Fan Yin, Di Wu et al.2023 · Conference on Empirical Methods in Natural Language Processing · 68 citations In Library
11Mistral 7BAlbert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch et al.2023 · arXiv.org · 3.7k citations In Library
12From task structures to world models: What do LLMs know?Ilker Yildirim, L. Paul, Ilker Yildirim2023 · Trends in Cognitive Sciences · 118 citations In Library

Scroll for more · 38 remaining

Similar papers

T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction TuningYanjun Fu, Faisal Hamman, Sanghamitra Dutta2025 · arXiv (Cornell University) · 10 citations In Library
Instruction Mining: Instruction Data Selection for Tuning Large Language ModelsYihan Cao, Yanbin Kang, Wang, Chi et al.2023 · arXiv (Cornell University) · 53 citations In Library
Cream of the Crop: Harvesting Rich, Scalable and Transferable Multi-Modal Data for Instruction Fine-TuningMengyao Lyu, Yan Li, Huasong Zhong et al.2025 · arXiv.org · 1 citation In Library
DONOD: Efficient and Generalizable Instruction Fine-Tuning for LLMs via Model-Intrinsic Dataset PruningJie Hu, Yang, Surong, Lijun Wu et al.2025 · arXiv (Cornell University) · 2 citations In Library
Automated Data Curation for Robust Language Model Fine-TuningJiuhai Chen, Jonas Mueller2024 · arXiv (Cornell University) · 36 citations In Library
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language ModelsYoumi Ma, Sakae Mizuki, Kazuki Fujii et al.2025 · arXiv.org · 3 citations In Library
A Framework for Fine-Tuning LLMs using Heterogeneous FeedbackRyan Aponte, Ryan A. Rossi, Shunan Guo et al.2024 · Recent Advances in Natural Language Processing · 1 citation In Library
Easy Dataset: A Unified and Extensible Framework for Synthesizing LLM Fine-Tuning Data from Unstructured DocumentsMiao, Ziyang, Qiyu Sun, Jingyuan Wang et al.2025 · arXiv (Cornell University) · 12 citations In Library
DELIFT: Data Efficient Language model Instruction Fine TuningIshika Agarwal, Krishnateja Killamsetty, Lucian Popa et al.2024 · arXiv (Cornell University) · 18 citations In Library
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction TuningMing Li, Yong Zhang, Zhitao Li et al.2023 · arXiv (Cornell University) · 373 citations In Library

Peer review

Accept (poster)NeurIPS 2024 · via OpenReview
Reviewer tj1t3/10 · confidence 4/52024-07-16

Summary

This paper explores the problem of Shapley-based data selection for instruction tuning. Specifically, the proposed approach is composed of three steps–clustering the target samples, Shapley-style evaluation for each clustering, and resampling from the clusters based on the Shapley scores. The paper validates its approach on two instruction tuning benchmarks and obtains favorable results against several baselines.

Strengths

The paper is clearly written and very easy to read. The problem is well-contextualized and well-formulated. The structure is complete and the language is neat. Evaluations are standard and a number of baselines are considered.

Weaknesses

The paper is probably a bit simple for a full publication of NeurIPS. It has concrete merits, though, I have concerns for several aspects of its contributions. 1. The paper adopts the expensive Shapley approach for data selection, citing its advantage for factoring in the combinatorial effects between samples. Yet, after estimating the Shapley score of the clusters, the final data selection is only conducted via resampling based on the score or ranking of the clusters, where the combinatorial effect between clusters is also not considered. In this sense, I am not sure whether using the costly Shapley approach delivers actual benefits. A grid search over the sampling ratio from each cluster could have better results. 2. Instruction mining has been "extensively studied" during the past year and a wealth of papers have emerged after the paper [LIMA: Less is more for alignment]. Most of those papers claim to reach comparable performance using just a few percentages of the original instruction tuning dataset. Yet, none of these papers are included as a baseline in this work. For the baselines compared within this work, DSIR is for pre-training data selection where its scalability and computation overhead and not at all comparable with the proposed method (and similarly for DQ). 3. It is very well known Shapley-style evaluations are notably expensive as they rely on repetitive model retrainings. It is important to benchmark the computation overhead for the proposed methods. 4. The paper claims the proposed method is suitable for any objectives. This very strong argument is not validated in this paper. For many tasks where this may not be true. For example, its computational complexity is prohibitive for pre-training data selection. Also, this approach only applies to cluster-level data selection but not to sample-level selection. Its effectiveness seems highly dependent on the embedding space for clustering. This work uses Sentence-Transfomer in the experiments. It may not work for other tasks. At least, these major limitations are not discussed in this paper. 5. As mentioned before, a notable number of methods have been proposed for the problem of instruction mining. This paper does not clearly identify a research gap to fill. A similar Shapley-style approach has been proposed TS-DSHAPLEY. Experiments conducted in this paper are also limited. In general, I have concerns about the novelty and contribution of this paper.

Questions

See Weaknesses.

Rating

3

Confidence

4

Soundness

2

Presentation

2

Contribution

1

Limitations

The paper claims the proposed method is suitable for any objectives. This very strong argument is not validated in this paper. For many tasks where this may not be true. For example, its computational complexity is prohibitive for pre-training data selection. Also, this approach only applies to cluster-level data selection but not to sample-level selection. Its effectiveness seems highly dependent on the embedding space for clustering. This work uses Sentence-Transfomer in the experiments. It may not work for other tasks. At least, these major limitations are not discussed in this paper.

Authorsrebuttal2024-08-12

Thank you for carefully reading our rebuttal and providing your feedback. **We greatly appreciate your recognition that our paper does not contain any technical flaws. However, we are concerned about the score of 3, which is typically reserved for papers with significant technical issues or weak evaluations.** Since you acknowledged there are no critical flaws, a score of 3 seems inconsistent with the paper's merit. We would like to further clarify the distinctions and advantages of SHED compared to TS-DSHAPLEY: TS-DSHAPLEY randomly samples multiple subsets of the training data, estimates the contributions of data points in these subsets, and then repeats the process multiple times and aggregates the results to estimate the Shapley values for the entire training set. Notably, this method will calculate the Shapley value for all instances in the training set. Advantages of SHED: - **Improved computational efficiency:** SHED only computes Shapley values for representative proxy data samples selected from clusters, rather than for each individual data point as in TS-DSHAPLEY. This dramatically reduces the computational overhead, making SHED more scalable to large datasets. - **Model-agnostic data representation:** SHED uses model-agnostic sentence embeddings (e.g., from Sentence Transformers) to represent data samples, while TS-DSHAPLEY relies on representations extracted from the target language model. The model-agnostic approach enhances the transferability of the curated datasets across different language models, amortizing the computational cost of data selection. Table 7 showcases the transfer performance of SHED-curated datasets across different models. SHED is trained on the MMLU and WizardLM datasets using the LLaMA-7B model, and the resulting optimal subsets are used to fine-tune various models, including LLaMA-13B, Vicuna-7B, and GPT-2. The results demonstrate that SHED-constructed datasets exhibit robust performance across a range of models, confirming their applicability across models and even different model families. This strongly supports the transferability of SHED datasets. - **Clustering-based data sampling:** SHED employs clustering algorithms to group similar data samples and selects representative proxy data for each cluster. This helps capture the diversity and complexity of the original dataset while reducing redundancy. TS-DSHAPLEY does not explicitly consider data diversity in its sampling process. - **Flexible optimization objectives:** SHED's value function in the Shapley value calculation can be customized for various optimization objectives, such as accuracy and fairness, allowing the curated datasets to align with task-specific requirements. TS-DSHAPLEY focuses primarily on model predictive accuracy. - **Unified and extensible framework:** SHED presents a unified framework that integrates clustering, Shapley value computation, and data sampling, with the flexibility to accommodate different clustering algorithms, optimization objectives, and sampling strategies. This makes SHED more adaptable to various scenarios than TS-DSHAPLEY. We hope this clarification helps to further understand the unique contributions and strengths of SHED, and we kindly request reconsideration of the score in light of this information. **The existence of a related but less effective method should not be grounds for rejection.** Thank you once again for your feedback and consideration.

Reviewer tj1t2024-08-13

Thanks for the response.

Thanks to the authors for the response. Let me explain where I feel this work can be improved. "Model-agnostic data representation" relies on the quality of embedding space. Compared with random sampling, the effectiveness of clustering also depends on the embedding quality. Sentence-Transformer often works quite nicely despite being task-agnostic. This has been witnessed in many works. But on a larger picture, how we move forward from here remains unclear. On tasks where Sentence-Transformer cannot capture the nuances, what are we going to do? "Clustering-based data sampling" is a standard practice for reducing computation complexity when the scale of the problem becomes an issue. For example, [Gio: Gradient information optimization for training dataset selection] also implemented the idea of clustering-based data selection. The work strives for "Flexible optimization objectives" and "Unified and extensible framework". But the development of this paper concentrates on "selecting instruction samples". First, selecting instruction samples is not a particularly computationally intensive task such that many methodologies can be applied. There is now a wealth of paper on this topic. Since the overall size of these instruction-tuning datasets is typically not very large, there has been a constant debate on whether research should focus on "selecting a subset of instruction-tuning examples". Given this being a crowded field, the work could greatly improve its impact if extended to broader applications beyond "instruction mining".

Authorsrebuttal2024-08-13

Thank you for your feedback

Thank you for your thoughtful feedback. We appreciate the points raised, and would like to address each one in turn: 1. Embedding Quality and Model-Agnostic Data Representation: - While there may be edge cases where Sentence-Transformer does not fully capture nuances, **SHED’s flexible framework allows for integrating alternative or task-specific embeddings, which can be chosen based on the task’s particular needs.** For example, CodeBert can be used to replace the Sentence-Transformer for tasks involving code understanding. In addition, if a task requires nuanced domain-specific understanding, fine-tuning the Sentence-Transformer to adapt to these needs can be implemented, like LLaVA's method, which only requires a small cost. This adaptability ensures that SHED remains effective across a diverse set of tasks, even beyond the current scope. - **However, if one insists on a truly "perfect" task-agnostic model, even widely-used powerful models like LLaMA3 cannot claim to be suitable for every possible task. The task-agnostic nature we discuss is within a reasonable and practical range.** 2. Clustering-Based Data Sampling: While clustering is commonly used to reduce computational complexity, **SHED’s innovation lies in considering combinatorial effects between data points, model-agnostic data selection, reducing computation cost, and a flexible framework.** Combining clustering with Shapley value calculations allows us to account for the combinatorial effects between data points, enhancing the selection process by considering their interactions. The transferability of SHED-refined datasets is the key advantage over other methods. Additionally, While classic Shapley value calculation is impractical, SHED significantly reduces computational costs by utilizing an approximation method and only calculating Shapley values for cluster proxies rather than individual data points, making it both efficient and scalable. Flexible framework design gives SHED the potential to be applied to various tasks. **We believe that utilizing well-established methods as a part of SHED should not be grounds for rejection.** 3. Importance of Instruction Selection - **The need for instruction-tuning data selection is particularly important as fine-tuning datasets grow increasingly large and complex.** For example, datasets like P3, which includes over 100 million examples, the Pile with 825 GB of text, and LAION-400M with 400 million image-text pairs, illustrate the scale at which fine-tuning can occur. Efficient, scalable methods like SHED are essential for managing such large datasets. **Besides, as the theoretical and experimental analysis shows, our approach is designed to scale to much larger datasets, proving its value in broader applications.** - **Furthermore, most researchers lack the resources to support pre-training from scratch, making fine-tuning an essential strategy.** SHED-selected datasets have demonstrated strong performance across multiple models, and releasing these curated datasets to the community could provide a significant contribution, helping researchers save substantial resources. - **The field of instruction-tuning data selection may indeed be crowded, but the crowdedness does not diminish the value of contributions that address real and pressing challenges. The computational advantages, transferability, and flexible design of SHED make it a valuable tool in this area.** Moreover, as instruction-tuning datasets continue to grow in size and complexity, the need for efficient, scalable data selection methods like SHED will only increase. By providing a framework that is both efficient and adaptable, we believe SHED makes a meaningful contribution to the field, with potential applications extending well beyond the current scope.

Reviewer Df9d6/10 · confidence 2/52024-07-16

Summary

The paper proposed a data refinement framework that refines datasets for fine-tuning LLMs by using the Shapley value. Based on the description, SHED is able to create smaller, high-quality datasets from large, extensive datasets without human intervention or commercial LLMs. This process involves three key components: model-agnostic clustering, proxy-based Shapley calculator, and optimization-aware sampling. Extensive experiments show that datasets curated by SHED achieve comparable or superior performance to full datasets while significantly reducing the data size and computational costs. Moreover, SHED-curated datasets exhibit strong transferability across various LLMs.

Strengths

1. The studied problem is critical as the proposed method is general to creat small yet high-quality datasets that benefit LLM finetuning in terms of performance improvement and computational cost reduction. 2. The experiments are extensive by covering multiple datasets and fine-tuning tasks.

Weaknesses

1. While SHED reduces the computational complexity of calculating Shapley values based on clustering and proxy-based calculations, it is unclear how effecient (e.g., in terms of time) it is compared to the classical calculation method. 2. As authods illustrated, the reliance on clustering may inadvertently reduce data diversity, which results in the overlook of rare but important samples.

Questions

1. Is it possible to evaluate SHED on a larger dataset in the industry level considering that the used datasets in the paper are still relatively small? 2. How will the selection of initial clustering groups affect the final model performance?

Rating

6

Confidence

2

Soundness

3

Presentation

2

Contribution

2

Limitations

No potential negative societal impact is observed.

Reviewer kWYC6/10 · confidence 3/52024-07-28

Summary

Tuning large language models to domain tasks is a difficult challenge requiring a dataset of high quality examples. Since noisy examples can significantly degrade performance, it is important to be able to curate these small, high-quality fine-tuning datasets. The authors propose filtering using each data point’s estimated contribution to the model performance using Shapley values. The proposed SHED method clusters the data, then iteratively deletes clusters based on the Shapley values of representative samples until a small dataset remains.

Strengths

- The topic of the paper is well-motivated. It is known that large models need high quality data during the fine-tuning process, and are sensitive to noisy data in this stage. High quality data can be prohibitively expensive to collect - Shapley values are a known and well-studied method to asses the importance of training points to a model’s predictions - Unlike previous data selection methods employing Shapley values (TS-DSHAPLEY), shed uses clustering and representative sample selection to significantly reduce the computational cost of data subsection. SHED also considers different target objectives (e.g. fairness), although it is not clear how easy it would be to adapt other methods to this objective. This is an interesting novelty that can help make shapley values more practically useful in this data selection setting. - Experiments are conducted on good language model eval datasets (e.g MMLU) - Results show that datasets filtered via SHED achieve improved performance compared to other sampling methods, and also compared to full dataset fine-tuning - Results are shown across a variety of evaluation datasets.

Weaknesses

- The related work leave room for improvement. In particular, the primary focus of the paper is a method that uses the estimated effect of each data point on accuracy to filter data points. Yet, the discussion of such related methods is lacking. Some recent works (e.g. DsDm https://arxiv.org/abs/2401.12926) also use the impact of individual data points to subselect datasets. - SHED uses a proxy data point (closest to the centroid) to estimate the predictive influence of the entire cluster. While this save on significant computational cost, the effectiveness of this stage depends significantly on the quality of the text embeddings used to compute similar, as well as the size and number of clusters - The experimental method training leaves room for improvement. All methods are stated to train for the same number of epochs and same other hyperparameters. However, as training can be sensitive to hyperparameters, it could make sense to conduct a sweep based on some val set for each method, and present results for the best hyperparameters on the test set. - It is not clear how the filtered data size was chosen. A variety of tables (e.g .Table 6 and 7) use different counts of QOCS and QWCS of 10k and 13k. Why not use the same ksubselected data size for all method for a clearer comparison?

Questions

Why are the different data selection methods compared with different final selected dataset sizes? How were the training hyperparameters chosen?

Rating

6

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

The authors have sufficiently discussed the limitations of their work.

Reviewer tj1t2024-08-12

Thanks for the responses.

Thanks for the responses. I have carefully read the rebuttal and other reviews. I appreciate the authors' effort, yet most of my opinions remain. I don't think there is a particular, critical flaw in the paper. It is a nice work that I would like to see in some venues, (e.g., as a short paper or at an ACL conference). Given the existence of similar works such as TS-DSHAPLEY, I don't think it is in the best interest to publish it at NeurIPS.

Reviewer Df9d2024-08-12

Thanks for the authors' response. They partially address my question related to the computation time of SHED. I have updated my scores.

Authorsrebuttal2024-08-13

Looking forward to your response

Dear Reviewer, As we are approaching the end of the rebuttal period, we would like to cordially inquire about the extent to which we have successfully addressed the concerns outlined in your review. Should any lingering points require further attention, please rest assured that we are enthusiastic about the opportunity to provide comprehensive responses to any subsequent queries or comments you may have. Your constructive input remains invaluable to us, and we appreciate your dedication to enhancing the quality of our manuscript. Thank you for your time and consideration. Best, Authors

Reviewer tj1t2024-08-13

Thanks for the prompt response

Thanks to the authors for the prompt response. My review remains unchanged.

Reviewer kWYC2024-08-14

Thank you for your response.

Thank you for your detailed response to my feedback. Please find an additional note below. W.1: Although Shapley does help consider interactions between data points, since you're re-computing the values after every cluster removal I don't think it actually matters here, i.e. compared to models that consider the contribution of each data point independently. Further, the claim that you are "capturing potential interaction effects between data points" is also not quite accurate, as the data points are grouped into larger clusters over which the Shapley values are computed.

Authorsrebuttal2024-08-14

Thank you for the feedback

Thank you for your insightful comment. We would like to address your concerns regarding the interaction effects between data points. The primary objective of removing clusters in our method is to calculate the marginal contribution, or boundary effect, of each cluster relative to the remaining clusters. By sequentially removing clusters and observing their impact on model performance, we can get how each cluster contributes to the overall model accuracy. After calculating the boundary contribution of a cluster in one iteration, we restart the process with a new randomized removal order. This repetition is crucial because it allows us to capture the variation in each cluster (or the data points within them)'s marginal contribution across different combinations of clusters. **By performing this process multiple times, with different sequences of cluster removal, we can average the different marginal contributions to accurately estimate the true Shapley value for each cluster. This allows us to observe how the presence or absence of a cluster in different cluster combinations influences the overall model performance, thereby capturing the interaction effects between the data points in different clusters.** **We agree with that the claim about capturing "interaction effects between data points" might not have been entirely accurate. It is more precise to state that our method considers the interaction effects between data points in different clusters. The clustering process itself is designed to group data points that are likely to interact in similar ways with the model and data points in other clusters.** By using Shapley values on these representative clusters, we capture a higher-level interaction effect, which is crucial for scaling the computation to larger datasets. While it’s not the same as evaluating every individual interaction, **this method strikes a balance between computational feasibility and the need to preserve interaction effects,** which we believe adds value to the data selection process. We sincerely appreciate your insightful feedback and will revise our manuscript to accurately reflect this aspect of our method. Your feedback is incredibly helpful in improving our paper.

Brando Miranda12025-03-31

Friendly Request for Reproducible Code

Hi Authors, I congratulate you on such interesting work! I really appreciated the idea of considering how examples interact (the main idea of your paper!) when selecting data. I was wondering, may I request a reproducible version of your code that is easy to run please? Thank you for considering my request. Sincerely, Brando Miranda PS: a quick Google search with "SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning Code" and related key words / phrases does not help me find a previous version of the code I once used I think. May I request the title of the repo be change to something more easily searchable please? --- Edit PS2: I found a link to the repo! https://github.com/Lucidreamer9/SHED-Shapley-Based-Automated-Dataset-Refinement/ Not sure why it was so hard to find.

Yexiao He12025-03-31

The code of SHED

You can find the code and dataset here: https://github.com/Lucidreamer9/SHED-Shapley-Based-Automated-Dataset-Refinement

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC