In knowledge graph construction, traditional NLP models suffer from poor cross-domain adaptability, while large language models (LLMs) face issues of hallucinations and data shift in information extraction. To address these challenges, we propose a collaborative framework for LLM-based triple extraction that integrates dynamic prompt generation and BERT assistance, effectively combining the entity recognition strengths of traditional models with the end-to-end reasoning capabilities of LLMs. On one hand, the framework uses the K-nearest neighbor (KNN) algorithm to vectorize text entity types. It matches highly relevant examples and generates dynamic prompts with progressive reasoning chain-of-thought (CoT) templates, enhancing LLMs' ability to adapt to task features and perform logical reasoning. On the other hand, it fine-tunes pre-trained BERT models to achieve accurate recognition and type annotation of text entities, alleviating LLM hallucinations by enhancing the structured representation of input text. Experimental validation on 3 cross-domain and cross-lingual datasets shows that when few-shot learning is combined with the CoT strategy, the triple extraction F1 score of Qwen2.5-14B increases from 0.1718 to 0.5856 (a 240.9 % relative improvement), demonstrating a performance breakthrough from strategy integration. After introducing BERT assistance, the F1 score of DeepSeek-V3 reaches 0.7426, and Qwen2.5-14B achieves a 277.3 % relative improvement compared to the zero-shot baseline, which verifies the effectiveness of cross-model collaboration. Compared with the traditional NLP model, it significantly enhances LLMs' knowledge transfer capability and domain adaptability. It provides an innovative paradigm for intelligent knowledge graph construction through the collaboration of traditional NLP techniques and prompt engineering, effectively compensating for the inherent limitations of LLMs.
Paper
Full text
A BERT-Assisted LLM Framework for Knowledge Graph Construction
OpenAlex · Advanced Graph Neural Networks · 2025
Abstract
In knowledge graph construction, traditional NLP models suffer from poor cross-domain adaptability, while large language models (LLMs) face issues of hallucinations and data shift in information extraction. To address these challenges, we propose a collaborative framework for LLM-based triple extraction that integrates dynamic prompt generation and BERT assistance, effectively combining the entity recognition strengths of traditional models with the end-to-end reasoning capabilities of LLMs. On one hand, the framework uses the K-nearest neighbor (KNN) algorithm to vectorize text entity types. It matches highly relevant examples and generates dynamic prompts with progressive reasoning chain-of-thought (CoT) templates, enhancing LLMs' ability to adapt to task features and perform logical reasoning. On the other hand, it fine-tunes pre-trained BERT models to achieve accurate recognition and type annotation of text entities, alleviating LLM hallucinations by enhancing the structured representation of input text. Experimental validation on 3 cross-domain and cross-lingual datasets shows that when few-shot learning is combined with the CoT strategy, the triple extraction F1 score of Qwen2.5-14B increases from 0.1718 to 0.5856 (a 240.9 % relative improvement), demonstrating a performance breakthrough from strategy integration. After introducing BERT assistance, the F1 score of DeepSeek-V3 reaches 0.7426, and Qwen2.5-14B achieves a 277.3 % relative improvement compared to the zero-shot baseline, which verifies the effectiveness of cross-model collaboration. Compared with the traditional NLP model, it significantly enhances LLMs' knowledge transfer capability and domain adaptability. It provides an innovative paradigm for intelligent knowledge graph construction through the collaboration of traditional NLP techniques and prompt engineering, effectively compensating for the inherent limitations of LLMs.