ChatGPT-like large language models for testing and verification of autonomous intelligent systems: a systematic review

Abstract

This paper provides a systematic review of how ChatGPT-like large language models (LLMs) contribute to the testing and verification of autonomous intelligent systems (AIS). Building upon recent advances in generative reasoning, this study integrates evidence from 120 peer-reviewed works to examine four key domains: test scenario generation, vulnerability detection, formal verification, and real-time monitoring. Comparative analysis across fuzz testing, symbolic execution, and reinforcement learning highlights how LLMs improve automation, semantic coverage, and adaptability while revealing limitations in benchmark completeness, interpretability, and resource efficiency. The review introduces structured tables summarizing representative datasets, domain-specific applications, and comparative insights between traditional and LLM-based testing approaches. Key challenges-including benchmarking gaps, explainability deficits, and ethical risks-are analyzed alongside emerging research directions such as hybrid verification frameworks and data quality enhancement. This work aims to bridge conceptual and practical gaps between AI safety engineering and large-model reasoning, offering a reference roadmap for integrating LLMs into future AIS verification pipelines.