Consistency Meets Verification: Enhancing Test Generation Quality in Large Language Models Without Ground-Truth Solutions

Large Language Models (LLMs) have significantly advanced automated test generation, yet existing methods often rely on ground-truth code for verification, risking bug propagation and limiting applicability in test-driven development. We present ConVerTest, a novel two-stage pipeline for synthesizing reliable tests without requiring prior code implementations. ConVerTest integrates three core strategies: (i) Self-Consistency (SC) to generate convergent test cases via majority voting; (ii) Chain-of-Verification (CoVe) for iterative, reasoning-guided code refinement; and (iii) a Dual Execution Agreement to cross-validate code and tests through consensus. Experiments on BigCodeBench and Less Basic Python Problems (LBPP) benchmarks demonstrate that ConVerTest improves test validity, line coverage, and mutation scores by up to ${3 9 \%, ~} {2 8 \%}$, and ${1 8 \%}$ respectively over baselines. Our findings highlight ConVerTest as a robust solution for mitigating hallucinations and enhancing the reliability of autonomous software testing agents.

Paper

References (52)

Scroll for more · 38 remaining

Similar papers

© 2026 NYSGPT2525 LLC