Can Models Help Us Create Better Models? Evaluating LLMs as Data Scientists

Response to Reviewer YFRZ (part 1)

We appreciate the time you've taken to review our manuscript and the opportunity your comments provide for us to clarify our work. Your feedback raises important points that we believe merit a thorough discussion. ## 1. Comparing to Existing Benchmarks and Literature Review ## You mention that our critique of existing benchmarks might be unfair, particularly because many of these benchmarks are well-recognized for testing base LLMs, while our benchmark seems tailored for post-trained LLMs. It's true that existing benchmarks are widely used—and for good reasons. They assess specific capabilities of language models, such as language understanding, code generation, or domain-specific knowledge recall. But that's precisely the point we're addressing. The issue isn't that these benchmarks are invalid; it's that they evaluate skills in **isolation**. Our contention is that to truly understand an AI's capabilities, especially in practical applications, we need *benchmarks that assess the integration of multiple skills*. Real-world problems rarely come neatly packaged, requiring only one type of expertise. They demand a combination of understanding, reasoning, and the ability to act upon that reasoning in a useful way. You suggest that comparing our benchmark to those designed for base models might be unfair. But if a base model can't integrate these skills, isn't that an important finding? The ability to combine competencies is a fundamental aspect of intelligence. If current base models lack this, it highlights an area where the field needs to improve. Regarding the vagueness in Section 1.1, perhaps we could have been more explicit. In our revised manuscript, we've expanded this section to include concrete examples illustrating how existing benchmarks fall short in assessing integrated skills. For instance: - **Practical Usability**: Benchmarks like COPA and HellaSwag focus on commonsense reasoning but don't translate into practical applications where solutions have real functional value. - **Complex Skill Integration**: Code generation benchmarks such as HumanEval assess coding ability in isolation and may not require the integration of extensive domain knowledge or reasoning skills. We delve deeper into our assessment in Appendix H, where each benchmark is discussed separately. Our aim isn't to dismiss these benchmarks but to highlight that they don't cover the integrated skill sets needed for tasks like feature engineering. If there's a specific benchmark you believe effectively assesses these integrated skills, we'd be interested to learn about it. Our literature review didn't find such benchmarks, but we're open to expanding our discussion if we've missed something. ## 2. Comparisons with Concurrent Works ## You mention that many concurrent works are testing models in complex, knowledge-intensive scenarios, and it's unclear how FeatEng compares to these benchmarks. Again, specific examples would be helpful here. Without knowing which works you're referring to, it's challenging to provide a direct comparison. We have conducted an extensive literature review and believe that FeatEng offers unique advantages: - **Unique Focus on Practical Application**: FeatEng specifically assesses the practical application of integrated skills in generating feature engineering code that directly impacts machine learning performance. - **Assessment of Skill Integration**: While other benchmarks might test complex reasoning or knowledge recall, FeatEng evaluates whether a model can apply its knowledge and reasoning to produce code that improves real-world data science tasks. If there are particular benchmarks you think are comparable, we'd welcome the opportunity to address how FeatEng relates to them. Our goal is not to claim that FeatEng is the only valuable benchmark but to show that it fills a gap in evaluating integrated, practical skills.

Respone to Reviewer KB4t (part 1)

We sincerely appreciate your detailed review and the valuable feedback you've provided. Your insights have helped us improve our manuscript, and we address your concerns point by point below. ## 1. Limited Result Interpretation #### a. Enhanced Visualization of Results Thank you for the suggestion regarding the visualization of our results. In the revised manuscript, we have included a new table with color-coded performance metrics to enhance readability and facilitate comparison across models. Additionally, we have provided line plots in the appendix (Appendix I, Figure 7, and Figure 8) to visually represent the correlation between FeatEng scores and Chatbot Arena ELO rankings. We believe these visual aids improve the clarity and interpretability of our results. #### b. Inclusion of Additional Baselines and Their Role We understand your suggestion to include more AutoML baselines for a comprehensive comparison. However, we believe that our current approach aligns best with the primary objectives of our study. We included a simple AutoML baseline as a point of reference to provide context for the performance of LLMs on the feature engineering task. This baseline serves to illustrate how LLMs compare to an established automated system in improving model performance through feature engineering. Our intention is not to benchmark AutoML systems but to demonstrate that LLMs can be competitive in this domain. #### c. Broader Takeaways Beyond Chatbot Arena While FeatEng can serve as a cost-effective alternative to Chatbot Arena for certain assessments, we believe its primary value lies in measuring the integration of multiple skills essential for advanced and versatile LLMs. - **Integration of Skills:** FeatEng assesses not only technical proficiency but also the ability to apply domain knowledge, reasoning, and code generation in practical, real-world scenarios. This integration of skills is central to LLM development and is perhaps underrepresented in current benchmarks. - **Correlation with User-Focused Metrics:** The strong correlation with user-focused metrics, despite not requiring conversational proficiency, underscores the benchmark's relevance. It suggests that FeatEng captures essential aspects of LLM capabilities that are important in practical applications. #### d. Impact of LLM's Number of Parameters We appreciate your interest in how model size affects performance on FeatEng. In the revised manuscript (Section 4.3), we have analyzed the relationship between generated feature types and performance, which provides insights about the task and model families. Regarding the model size and the Claude family, there is an interesting finding from Table 1: we observed that while larger models like **Claude-3-Opus** generally perform better, they also exhibit more variability, often failing to improve upon the baseline. Interestingly, when these models do improve, they tend to do so by a significant margin. This suggests that factors such as training data diversity, model architecture, and sampling strategies play significant roles alongside model size. ## 2. Comparison Between Single-Pass LLM Evaluation and Iterative AutoML Algorithms You raise an important point regarding the comparison between single-pass LLM evaluations and iterative AutoML algorithms. - **Different Operational Contexts:** Our aim is to evaluate each method in its native operational context. AutoML systems are designed for iterative optimization, refining models through cycles of adjustments. In contrast, LLMs currently generate solutions in a single pass, especially when provided with an example solution. - **Value of Comparison:** Comparing the two approaches provides valuable insights into their respective strengths and weaknesses. Our findings demonstrate that LLMs, even without iterative refinement, can produce meaningful feature transformations that enhance model performance. This sets a baseline for their current capabilities in feature engineering tasks. - **Future Directions:** We acknowledge that integrating LLMs into iterative frameworks, similar to approaches in CAAFE [3], could potentially enhance their performance. This represents an exciting direction for future research. Our benchmark can serve as a foundation for such developments by providing an initial assessment of LLMs' abilities in a single-pass context. ## 3. Clarity of Figures Thank you for this suggestion. We agree that a flowchart can provide a clearer understanding of the evaluation pipeline. - **Updated Figure:** In the revised manuscript, we have replaced Figure 3 with a chart (now Figure 4) that illustrates how the components of the benchmark are structured and interact. ## Conclusion Your insights have been instrumental in enhancing the clarity and depth of our manuscript. We have addressed each of your concerns point by point, and we believe that the revisions we've made strengthen our work significantly.

Single- vs Multi-pass Evaluation Setup

Thank you once again for your time and valuable insights. The primary objective of this paper is to introduce a data set we created and to argue that the problem we propose has some desired characteristics that existing problems lack. The single-step generation we used to evaluate existing models is the simplest one we can get while tackling the problem. From our point of view, considering multi-step, agentic workflows (or methods such as CAAFE) would require either sacrificing fairness of comparison or extensive optimization of workflows for each model separately. Consequently, we decided to focus on the most straightforward, meaningful approach while proposing the FeatEng problem to be able to execute it and describe it faithfully in a single paper. Though human-like, iterative work on the data set would lead to better results, our setup is reasonable. One way to back up this claim is to consider our correlation with Chatbot Arena ELO. The FeatEng's canonical setup that correlates with ELO more than MMLU, ARC Challenge, BBH, and even Arena-Hard has to be at least a reasonable approximation (see Appendix G and Figure 7). Nevertheless, the canonical method presented in the paper is one of many ways to use the data set we introduce. We do not constrain the potential solution regarding how it approaches solving this problem, and we anticipate researchers will also consider FeatEng in more interactive approaches that gradually improve the solution. These are possible, e.g. by splitting the train DataFrame to obtain the dev set on which the agent can operate. We hope this addresses your concern; we will clarify this topic in the revision.

Response to Reviewer Eugm (part 1)

We appreciate your thoughtful review and the opportunity to clarify and strengthen our work. Your insights have prompted us to delve deeper into the nuances of our benchmark and address the concerns you've raised. We also modified our submission in order to account for the analysis of how successful models approach our benchmark. ## 1. Potential Overfitting and Memorization You make a great point about models potentially overfitting to specific types of data or common data science tasks, especially if they've been trained on similar datasets. This is a real challenge when trying to assess the true abilities of large language models (LLMs). However, feature engineering is naturally open-ended. The effectiveness of particular features can vary widely depending on the dataset and the problem. A feature that works well in one situation might be useless or even harmful in another. This variability makes it hard for models to rely only on memorized techniques. ### How FeatEng Addresses Overfitting - **Diverse Datasets:** We use 103 datasets from various fields like healthcare, finance, and entertainment (see Section 3.1 and Figure 2 in our revised manuscript). This variety forces models to adapt their strategies instead of using a one-size-fits-all approach. - **Unique Problem Descriptions:** Each dataset includes specific instructions and target variables, requiring models to understand and process new information instead of just recalling what they've seen before. Moreover, we have rewritten the task description, and feature description to be concise, informative, and unified. - **Focus on Contextual Understanding:** Success in our benchmark relies on the model’s ability to combine relevant domain knowledge and create new features or remove unnecessary ones. Moreover, while models may have encountered similar data during training, the open-ended nature of feature engineering demands adaptability and creativity. There's always room for new insights and improvements, which means that models can't simply rely on memorization if they aim to excel. In our revised manuscript (Section 3.2), we delve deeper into successful feature engineering. We found that top-performing models go beyond basic data cleaning by leveraging domain knowledge and creativity to generate meaningful, contextually relevant features. This demonstrates that the benchmark demands LLMs to effectively integrate multiple skills to adapt to real-world data science challenges. ## 2. Dependence on XGBoost as the Sole Evaluation Model Thank you for suggesting the use of multiple evaluation algorithms. We chose **XGBoost** because it is robust, efficient, and widely used in real-world applications. Given the scale of our benchmark (over 100 datasets), we needed an algorithm that offers fast training and evaluation times without compromising performance. Moreover, it handles different data types well and doesn’t require extensive feature scaling, making our evaluation straightforward. As of now, our initial focus remains on simplicity. Using a single, well-understood model helps maintain consistency across evaluations. It allows us to attribute performance differences directly to the feature engineering efforts rather than variations in the learning algorithms. We agree that expanding the evaluation to include other models is a valuable direction for future work. ## 3. Resource Intensity of Running the Benchmark You raised a valid concern about the computational resources needed to run FeatEng, especially with larger datasets or complex transformations. We have designed the benchmark to balance realism with accessibility. Most modern systems can efficiently handle the computations. For instance, models like **Llama-3-405B** completed our benchmark in approximately seven minutes on standard hardware (8x H100), processing 20 examples for each of the 103 datasets by utilizing **vllm** and prefix caching. To make FeatEng more accessible, we offer per-dataset results so users can work with smaller data subsets if needed, reducing the computational load. Compared to methods that require extensive human involvement, our automated benchmarking is a cost-effective and scalable alternative. ## 4. Absence of a Human Baseline Including a human baseline is important for comparing model performance. However, creating human benchmarks for our diverse tasks would require significant effort, as many of our datasets don’t have existing human performance metrics. This stems from the fact that we searched over less popular datasets on Kaggle, rather than over Kaggle competitions. In our revised manuscript (Section 6), we propose involving the data science community to contribute human-generated solutions. This collaborative approach will help us gather human baselines over time and provide valuable context for our benchmark results.

Response to Reviewer nK3S (part 1)

# Response to Reviewer Thank you for your thoughtful review and the opportunity to engage with your concerns. Your feedback highlights important considerations that deserve a thorough response. I'd like to address your points by focusing on three main areas: the scope and significance of feature engineering as a benchmark, the role of coding abilities in evaluating large language models (LLMs), and how challenging the benchmark is. ## 1. The Scope and Significance of Feature Engineering You mention that "writing feature engineering code" is a narrow application and not a frequent use case for LLMs, suggesting it might not reflect all fundamental aspects of intelligence. I understand why it might seem that way at first glance. Feature engineering is indeed a specialized task within the field of data science. However, I believe that its complexity and the range of skills it requires make it a meaningful benchmark for assessing broader intelligence in LLMs. Consider feature engineering not just as a coding exercise, but as a synthesis of multiple cognitive abilities: - **Domain Knowledge Application:** Effective feature engineering demands an understanding of the specific domain—be it healthcare, finance, or any other field. This requires the model to recall and apply relevant facts and concepts. - **Reasoning and Problem-Solving:** Deciding which features to create involves logical reasoning and the ability to identify patterns or relationships within the data. - **Creativity and Innovation:** Crafting new features often requires creative thinking to combine existing data in novel ways that can improve model performance. - **Technical Skills:** Translating these ideas into executable code necessitates proficiency in programming. In this sense, feature engineering serves as a microcosm of complex problem-solving tasks. It's not merely about following a set of predefined steps; it's about **integrating diverse skills** to achieve a practical goal. Think of it like composing music. Not everyone does it, and it's a specialized skill, but it requires a deep understanding of music theory (domain knowledge), creativity, emotional intelligence, and technical proficiency with instruments or software. Evaluating an LLM's ability to compose music would tell us a lot about its multifaceted intelligence, but there is a slight issue with building an automated benchmark for that. While it might not be a daily use case for all users, the skills required for feature engineering are representative of the broader capabilities we expect from intelligent systems. ## 2. The Role of Coding Abilities in Evaluating LLMs You make a valid point that not all "strong LLMs" are built for coding, and our benchmark might favor those with coding skills over other types of intelligence. While coding isn't the sole measure of intelligence, in many real-world applications, implementing solutions is as crucial as understanding problems. For LLMs, coding bridges comprehension and action, demonstrating not just syntax proficiency but also deep understanding and practical application. Regarding the concern that our evaluation might prioritize coding LLMs over "truly intelligent models," I'd argue that *the integration of coding skills with other forms of intelligence is itself a valuable measure*. An LLM capable of reasoning but unable to act on that reasoning is less useful in practice. For instance, models like **codestral-22B** do well not only because they can code but also because they effectively combine coding with domain knowledge and reasoning, enhancing performance through feature engineering. ## On the Challenge and Flexibility of the Metric You express concern that feature engineering might involve only a handful of strategies, potentially limiting the challenge and flexibility of the benchmark. In practice, feature engineering is a rich and nuanced field. While there are common techniques like one-hot encoding or normalization, the most impactful feature engineering often comes from deep domain knowledge and creative problem-solving. Our analysis revealed that top-performing models didn't just apply standard preprocessing steps. They: - **Enriched Datasets with External Knowledge:** Such as mapping countries to continents or incorporating economic indicators. - **Created Interaction Terms:** Capture complex relationships between variables, like combining biomarkers in healthcare datasets to predict patient outcomes. - **Identified and Transformed Features in Innovative Ways:** Demonstrating creative thinking beyond obvious strategies. These tasks require more than applying a set of predefined strategies. They demand that the model understands the context and can devise novel solutions tailored to each specific dataset. We have significantly expanded the qualitative and quantitative analysis of the model’s outputs in the new revision. Please see section 3 for a more detailed description.

Appendix to Summary

Following further discussion we extend the summary with responses and document changes introduced recently. We hope the high level of discussion activity didn’t cause you to spend significantly more time than planned on this paper. Your insights have been invaluable in improving our work, and we deeply appreciate your effort. _____ ### 8. Other Efforts to Mimic ChatbotArena Scores (YFRZ) **Concern:**\ Other works, such as MixEval, aim to mimic ChatbotArena scores cost-efficiently. How does FeatEng compare to them? **Response:**\ We analyzed the correlation between FeatEng scores and Chatbot Arena ELO using MixEval's methodology. These new results included in **Appendix G** and **Figure 7** show our benchmark on the Pareto frontier with MixEval-Hard, even though FeatEng wasn't designed to align with human judgment. _____ ### 9. Single- vs Multi-pass Evaluation (KB4t) **Concern:**\ How can we assess the model's capability when considering a single pass, unlike real Kaggle competitors, who iteratively develop the best-performing solution? **Response:**\ This paper introduces a new dataset and argues that the proposed problem offers unique advantages over existing benchmarks. To ensure fairness and simplicity, we evaluated models using single-step generation, avoiding the complexity of multi-step workflows. Despite this, the setup correlates strongly with Chatbot Arena ELO, validating its reasonableness. While the single-pass method is emphasized, we encourage researchers to use the FeatEng dataset with more interactive or agentic approaches. _____ This concludes the rebuttal process on our side.