Towards Optimizing with Large Language Models

In this work, we conduct an assessment of the optimization capabilities of LLMs across various tasks and data sizes. Each of these tasks corresponds to unique optimization domains, and LLMs are required to execute these tasks with interactive prompting. That is, in each optimization step, the LLM generates new solutions from the past generated solutions with their values, and then the new solutions are evaluated and considered in the next optimization step. Additionally, we introduce three distinct metrics for a comprehensive assessment of task performance from various perspectives. These metrics offer the advantage of being applicable for evaluating LLM performance across a broad spectrum of optimization tasks and are less sensitive to variations in test samples. By applying these metrics, we observe that LLMs exhibit strong optimization capabilities when dealing with small-sized samples. However, their performance is significantly influenced by factors like data size and values, underscoring the importance of further research in the domain of optimization tasks for LLMs.

Paper

References (11)

09Can llms generate random numbers? evaluatingllm sampling in controlled domains2023
10LLM : (ˆy1, ˆy2, ˆy3) = [1 , 1 , 1]
112023. Context-faithful prompting for large language modelsArXiv, abs/2303

Similar papers

© 2026 NYSGPT2525 LLC