ForgeBench: A Machine Learning Benchmark Suite and Auto-Generation Framework for Next-Generation HLS Tools

High-Level Synthesis (HLS) has gained traction in hardware design, yet two key limitations prevent it from becoming mainstream. ❶ First , existing HLS benchmarks are outdated and unrepresentative of modern machine learning (ML) workloads. Legacy suites such as MachSuite [1] and Rosetta [2] rely on outdated algorithms, while Rodinia [3] and Poly-Bench [4] were designed for GPUs and software compilers, respectively, and thus offer limited relevance to HLS evaluation. Even recent efforts like HLSyn [5] exclude modern architectures such as Large Language Models (LLMs). Meanwhile, the research community increasingly evaluates HLS tools using advanced ML models through ad-hoc, private implementations, making fair and reproducible comparisons impossible without a standardized, open-source benchmark. ❷ Second , current HLS tools are accelerator-oriented , targeting simple standalone functions and generating one-off accelerators via static dataflow analysis. However, modern ML hardware design is architecture-oriented , requiring general-purpose architectures with flexible dataflow control, multi-application mapping, and complex design hierarchies featuring shared module extraction and reuse. This mismatch between tools and design practice limits HLS productivity. To address these limitations, we propose ForgeBench , an ML-centric HLS benchmark suite and generation framework for next-generation HLS tool design and evaluation. Our key contributions are: (1) an open-source, extensible HLS design generation framework supporting easy integration of new designs and applications; (2) over 10,000 high-quality, diverse ML-representative HLS benchmarks that expose limitations of modern HLS frameworks on ML workloads; and (3) an architecture-oriented benchmark suite featuring pairs of HLS designs with manually implemented module-level reuse and mappings, serving as a baseline for evaluating future modular HLS tools. We demonstrate the diversity of our generated designs across latency and resource usage, and show the potential for module reuse within HLS designs. ForgeBench is open-sourced at https://github.com/hchen799/ForgeBench .