VERIFYRAG: A Dual-Layer Hallucination Detection Framework Combining Semantic Consistency Scoring and NLI-Based Evidence Grounding For Open-Domain LLMS
DOI:
https://doi.org/10.67440/ahj.vi.2790Keywords:
Hallucination Detection, Large Language Models, Retrieval-Augmented Generation, Natural Language Inference, Semantic Consistency, DeBERTa, Sentence-BERT, FAISS.Abstract
Large Language Models (LLMs) are being used more and more in knowledge-intensive tasks, yet they often provide outputs that are factually inaccurate with high confidence, a phenomenon known as hallucinated content. Current methods have little practical utility because they either address detection and mitigation separately or rely on costly proprietary APIs. VerifyRAG, a lightweight, open-source pipeline for detecting and mitigating hallucinations that incorporates a unique dual-layer verification architecture, is proposed in this study. Semantic consistency is used in Layer 1. Semantic alignment between LLM-generated replies and retrieved contextual data is scored using Sentence-BERT. Using DeBERTa-v3, Layer 2 employs Natural Language Inference (NLI)-based Evidence Grounding to categorise each assertion as Entailed, Neutral, or Contradicted in relation to retrieved documents. When the risk of hallucinations is high, a confidence scoring engine that fuses the two layers automatically corrects the response.The system is tested using the TriviaQA, HaluEval, and FEVER benchmarks and is based solely on open-source models. The findings of the experiment show a considerable decrease in the rate of hallucinations while preserving competitive response quality, providing a repeatable and economical approach for the implementation of LLM in the real world.
