Large language models (LLMs) have demonstrated remarkable success across diverse natural language processing (NLP) tasks, yet their limited knowledge reserves and persistent hallucinations undermine performance in complex reasoning. Knowledge graphs (KGs), with structured and verified information, provide a reliable foundation for reasoning, but existing KG-based LLM methods often treat KGs as static bases, neglecting their structural information. This oversight introduces spurious knowledge and compromises answer accuracy. To address these limitations, we propose LRwP, i.e., knowledge graph-enhanced Large language model Reasoning with Prompt engineering framework, which synergistically integrates LLMs and KGs while enhancing prompt design to improve reasoning fidelity and interpretability. LRwP comprises two key stages: (1) Subgraph retrieval, where a refined Personalized PageRank algorithm aligns queries with KG structures to yield compact subgraphs with maximized answer coverage; and (2) Reasoning, where task-specific prompts guide LLMs to generate chains of thought and KG paths, which are semantically and structurally aligned to identify the most relevant answers. Extensive experiments demonstrate that LRwP significantly outperforms state-of-the-art baselines on both simple and multi-hop reasoning tasks, delivering more faithful and interpretable results.
Paper
Full text
Knowledge Graph-Enhanced Large Language Model Reasoning with Prompt Engineering
Semantic Scholar · 2025
Abstract
Large language models (LLMs) have demonstrated remarkable success across diverse natural language processing (NLP) tasks, yet their limited knowledge reserves and persistent hallucinations undermine performance in complex reasoning. Knowledge graphs (KGs), with structured and verified information, provide a reliable foundation for reasoning, but existing KG-based LLM methods often treat KGs as static bases, neglecting their structural information. This oversight introduces spurious knowledge and compromises answer accuracy. To address these limitations, we propose LRwP, i.e., knowledge graph-enhanced Large language model Reasoning with Prompt engineering framework, which synergistically integrates LLMs and KGs while enhancing prompt design to improve reasoning fidelity and interpretability. LRwP comprises two key stages: (1) Subgraph retrieval, where a refined Personalized PageRank algorithm aligns queries with KG structures to yield compact subgraphs with maximized answer coverage; and (2) Reasoning, where task-specific prompts guide LLMs to generate chains of thought and KG paths, which are semantically and structurally aligned to identify the most relevant answers. Extensive experiments demonstrate that LRwP significantly outperforms state-of-the-art baselines on both simple and multi-hop reasoning tasks, delivering more faithful and interpretable results.