Misinforming LLMs: vulnerabilities, challenges and opportunities

Large Language Models (LLMs) have made significant advances in natural language processing, but their underlying mechanisms are often misunderstood. Despite exhibiting coherent answers and apparent reasoning behaviors, LLMs rely on statistical patterns in word embeddings rather than true cognitive processes. This leads to vulnerabilities such as"hallucination"and misinformation. The paper argues that current LLM architectures are inherently untrustworthy due to their reliance on correlations of sequential patterns of word embedding vectors. However, ongoing research into combining generative transformer-based models with fact bases and logic programming languages may lead to the development of trustworthy LLMs capable of generating statements based on given truth and explaining their self-reasoning process.

Paper

References (12)

07Don’t Say No: Jailbreaking LLM by Sup-pressing Refusal2024 · arXiv preprint
082022. Chain-of-thoughtpromptingelicitsreasoning in large language modelsAdvances in neural information processing systems
092024. ChatGPT is bullshitEthics and Information Technology
102024. Thought-Like-Pro: Enhancing Reasoning of Large Lan-guageModelsthroughSelf-DrivenProlog-basedChain-of-ThougharXivpreprint
112024. GRAG: Graph Retrieval-Augmented GenerationarXiv
122023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language modelsarXiv preprint

Similar papers

© 2026 NYSGPT2525 LLC