Experimental evidence of progressive ChatGPT models self-convergence

Large Language Models (LLMs) that undergo recursive training on synthetically generated data are susceptible to model collapse, a phenomenon marked by the generation of meaningless output. Existing research has examined this issue from either theoretical or empirical perspectives, often focusing on a single model trained recursively on its own outputs. While prior studies have cautioned against the potential degradation of LLM output quality under such conditions, no longitudinal investigation has yet been conducted to assess this effect over time. In this study, we employ a text similarity metric to evaluate different ChatGPT models'capacity to generate diverse textual outputs. Our findings indicate a measurable decline of recent ChatGPT releases'ability to produce varied text, even when explicitly prompted to do so, by setting the temperature parameter to one. The observed reduction in output diversity may be attributed to the influence of the amounts of synthetic data incorporated within their training datasets as the result of internet infiltration by LLM generated data. The phenomenon is defined as model self-convergence because of the gradual increase of similarities of produced texts among different ChatGPT versions.

Paper

References (8)

04Strong Model Collapse (No. arXiv:2410.04840)2024 · arXiv
05Retain for all models and temperatures the same prompt
06For every original text, temperature and model, create paraphrases with the ChatGPT API
07Identify all common patterns among original and ChatGPT paraphrases for several pattern lengths
08One paraphrase per text should be created per round with time gap, to avoid possible memory/cache effects by the ChatGPT API

Similar papers

© 2026 NYSGPT2525 LLC