Thank you for your valuable comments and for taking the time to review our paper. We are glad that some of your concerns have been addressed, and we appreciate the opportunity to clarify and expand on the additional points.
***
> Q1. Prompt optimization
A1. We appreciate the suggestion, but there are significant differences between the typical applications of these methods and the challenges of synthetic tabular data generation.
Self-discover and OPRO are designed to enhance LLM accuracy in structured tasks like multiple-choice questions or math problems, where correctness is relatively explicit. However, synthetic tabular data cannot be directly evaluated for correctness, making these methods less applicable to our domain.
Recognizing the potential, we adapted OPRO, which provides official code, to our task by optimizing prompts based on the classification accuracy of a robust classifier, CatBoost. Using the Thyroid dataset with GPT-3.5-turbo, we applied OPRO to optimize our prompt. Unfortunately, **the optimized prompts failed to produce valid synthetic tabular data**. For instance, one generated prompt was:
`No,45,F,No,No,No,Euthyroid,Single nodular goiter-right,No,Papillary,Multi-Focal,Intermediate,T3a,N1b,M0,II,Excellent`
This is because OPRO is designed to optimize simple instructions, e.g., `Take a deep breath and work on this problem step-by-step` or `Break this down`, as shown in Table 1 of the OPRO paper. **While effective for question-answering and reasoning tasks, these types of prompts are unsuitable for constructing the complex structures required in synthetic tabular data generation.**
Our experiments indicate that our proposed approach remains the most effective method for generating high-quality synthetic tabular data. Future research could explore prompt optimization methods specifically designed for tabular data generation to further enhance performance.
***
> Q2. Dataset details & Generalizability
A2. As noted in Section 3, we provided dataset details in Appendix I.2. However, **we acknowledge that this information may not be sufficiently highlighted in the main text**.
To address this, **we will revise the manuscript to include a comprehensive introduction to these datasets within the main text, clearly demonstrating the broad applicability and generalizability of our method.** We will also emphasize the importance of tabular data research in enhancing decision-making and efficiency in various real-world applications, better communicating the impact and significance of our work in multiple fields.
***
> Q3. Advanced classifier for label matching
A3. We employed top-performing tabular classifiers, XGBoost, CatBoost, LightGBM, and Gradient boosting classifier, known for their strong performance, often surpassing recent deep learning models. These models have served as robust baselines in tabular classification (TabR, ICLR’24 [1]) and have been used to assess generated data quality in tabular generation studies (the benchmark paper, NeurIPS’23 [2] and TabDDPM, ICML’23). In our work, we evaluated label matching quality by averaging results across 20 runs with these models, five runs each, demonstrating the superiority of our method.
We conducted preliminary experiments using recent in-context learning-based tabular classification methods, TabPFN (ICLR’23) [3] and T-Table (KDD’24) [4], on Travel, as detailed in **Table B of the attached PDF**. We also tested TabR (ICLR’24) [1], an advanced deep-learning tabular classification model, using its official code.
|Model|Original|+Ours|+TabDDPM|+GReaT|
|-|-|-|-|-|
|TabR|46.41|**60.78**|44.88|32.41|
The F1 score results indicate that **while the advanced models like TabPFN, T-Table, and TabR did not outperform traditional classifiers, adding synthetic data generated by our method consistently led to significant performance improvements, even with advanced tabular classification models.** In contrast, baselines resulted in performance decreases. Our method uniquely and consistently enhanced the performance of different classifiers, demonstrating superior label matching quality.
Furthermore, as discussed earlier, our label matching quality is validated, regardless of the classifier, by:
* Distinct class distribution in the generated data (Fig. 1, Toy set)
* Feature correlation similarity with the original data across classes (Fig. 4, Travel & Fig. 5, Sick)
These findings affirm that our model produces data with the best label matching compared to other baselines.
***
[1] TabR: Tabular Deep Learning Meets Nearest Neighbors, ICLR’24
[2] Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmark, NeurIPS’23
[3] TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second, ICLR’23
[4] From Supervised to Generative: A Novel Paradigm for Tabular Deep Learning with Large Language Models, KDD’24
[5] Self-consistency improves chain of thought reasoning in language models, ICLR’23