VeriReason: Reinforcement Learning with Testbench Feedback for Reasoning-Enhanced Verilog Generation

Automating Register Transfer Level (RTL) code generation with Large Language Models (LLMs) can reduce manual hardware design effort. However, current LLM-based approaches face four challenges: limited availability of high-quality training data, weak alignment between natural language specifications and generated code, lack of built-in verification mechanisms, and difficulty in adapting general-purpose models to RTL-specific constraints. Inspired by DeepSeek-R1, which combines reinforcement learning with reasoning capabilities, we introduce VeriReason, a framework that integrates supervised fine-tuning with Group Relative Policy Optimization (GRPO) for RTL code generation. Using high-quality training examples, a feedback-driven reward model, testbench evaluation, and structural heuristics, VeriReason improves specification-code alignment, reduces hallucinations, and strengthens reasoning traces and first-attempt functional correctness. To our knowledge, VeriReason is the first system that successfully integrates explicit reasoning capabilities with reinforcement learning for Verilog generation. On VerilogEval-Machine, VeriReason reaches 83.1% pass@5, while consistently outperforming comparable-sized open-source models. Our approach demonstrates up to a 2.8 × increase in first-attempt functional correctness compared to baseline methods.