Large language models produce text that reads well but can fabricate facts with the same fluency as accurate statements. This hallucination problem is a deployment blocker in medicine, law, and science. Varun Teja Chundru and Debasmita Biswas from Purdue University Fort Wayne tackle it from three angles: detection, mitigation, and domain transfer. Their system combines a fine-tuned classifier, uncertainty quantification, and calibration into a single pipeline, then uses the detector's signal to train a generator that hallucinates less.

Why Single-Signal Detectors Fall Short

The hallucination detection problem maps naturally onto natural language inference: given a knowledge source, a prompt, and a generated response, decide whether the response is faithful or fabricated. Prior work has explored entailment classifiers and retrieval-based verification, but two gaps persist. First, a single forward pass through a classifier produces a point estimate with no confidence signal, making it impossible to distinguish high-certainty detections from borderline cases. Second, detectors trained on general-domain data degrade sharply on specialized domains where terminology and reasoning patterns differ.

The authors address both gaps. They build a pipeline that produces three inference signals from a single DeBERTa-v3 backbone: a standard single-pass probability, a Monte Carlo Dropout mean and variance across 20 stochastic passes, and a temperature-scaled calibrated probability. Two ensemble methods combine these signals. The pipeline is evaluated on the HaluEval benchmark (30,000 samples across QA, Dialogue, and Summarization), on cross-domain transfer to the SciFact biomedical dataset, and in a closed-loop mitigation experiment where the detector drives DPO training of a generator.

Building the Three-Signal Detection Pipeline

The core classifier is DeBERTa-v3-base, which uses a disentangled attention mechanism that separates content and position representations into distinct vectors. Attention weights are computed using disentangled matrices for content-to-content, content-to-position, and position-to-content interactions, and an enhanced mask decoder aggregates these signals. For hallucination detection, the model receives a concatenated input of the knowledge context, the prompt, and the generated response, and outputs the probability that the response is hallucinated. Training uses AdamW with learning rate 2e-5, linear warmup over 10% of steps, batch size 16, and 3 epochs in fp32 mixed precision.

A single forward pass through this classifier is overconfident near the decision boundary, where small logit shifts produce large probability swings. The authors keep dropout active at inference time and run 20 stochastic forward passes. The mean across passes serves as the prediction, and the standard deviation captures epistemic uncertainty. Averaging across these stochastic sub-networks reduces variance and smooths probabilities near the boundary, improving top-1 accuracy while producing an uncertainty signal that peaks on genuinely ambiguous inputs.

Temperature scaling provides the third signal. A scalar temperature is learned on the validation set by minimizing negative log-likelihood: dividing logits by this temperature before softmax sharpens or flattens the distribution without changing the argmax, so accuracy and F1 remain unchanged but probabilities become better calibrated. This matters because uncalibrated MC Dropout variance is suppressed on uncertain examples and negative log-likelihood is inflated on wrong-but-confident predictions, causing threshold instability across domains.

Two ensemble methods combine these signals. Simple Average takes the unweighted mean of the single-pass and MC Dropout probabilities. The LR Meta-Classifier trains a logistic regression on four validation-set features: single-pass probability, MC Dropout mean, MC Dropout standard deviation, and cosine similarity between MiniLM embeddings of context and response.

What the Numbers Show on HaluEval

The main results on HaluEval's 4,500-sample test set tell a clear story. A zero-shot DeBERTa-MNLI model without any fine-tuning achieves only F1=0.430 and AUROC=0.650, confirming that off-the-shelf NLI transfer is inadequate. Fine-tuning on HaluEval brings the single-pass model to F1=0.915 and AUROC=0.977. MC Dropout inference pushes accuracy from 91.3% to 93.2% with no additional training, a 1.9-point gain concentrated near the decision boundary. The Simple Average ensemble achieves the highest AUROC at 0.979, while the LR Meta-Classifier matches MC Dropout on accuracy (0.931) but loses AUROC (0.960) because the retrieval similarity feature, cosine similarity between context and response embeddings, achieves only AUROC of roughly 0.38 in isolation. Faithful and hallucinated responses share surface vocabulary in HaluEval, making this near-random feature a drag on ranking quality.

Per-task breakdowns reveal that QA is easiest (F1=0.97) because hallucinated factoid answers typically substitute incorrect entities or numbers detectable from the question alone. Summarization follows at F1=0.96, given the source document as reference. Dialogue is hardest at F1=0.82 because conversational responses are shorter, more implicit, and contain fewer lexical anchors to the knowledge source.

Proving the Detector Reasons, Not Memorizes

A critical question for any NLI-based detector is whether it performs genuine entailment reasoning or exploits surface patterns. The authors strip the knowledge context from all test inputs, keeping only the prompt and response, and re-evaluate the same fine-tuned weights. Overall F1 drops from 0.91 to 0.82, confirming that the model leverages the knowledge source. The effect is task-dependent and revealing. Summarization F1 drops 24%, from 0.96 to 0.73, because detecting hallucinated summaries requires comparing against the source document by definition. QA F1 drops only 1%, from 0.97 to 0.96, because factoid hallucinations are often detectable from the question-answer pair alone: an implausible entity substitution stands out without re-reading the passage. Dialogue sits in between, dropping from 0.82 to 0.79.

How Little Training Data You Actually Need

Learning curve analysis reveals a sharp data-efficiency elbow. At 10% of the training data (2,100 examples), the model fails entirely with F1=0.01. At 25% (5,300 examples), F1 jumps to 0.70, capturing 77% of full-data performance. Performance continues improving to 0.82 at 50% and 0.95 at 100%, but with diminishing returns. For practitioners deploying hallucination detection in a new domain, this means bootstrapping a usable detector requires only around 5,000 labeled examples, not the full 21,000 used in this study.

Using the Detector to Train a Less Hallucinating Generator

The strongest test of a detector's utility is whether it can improve a generator. The authors apply Direct Preference Optimization to Qwen2.5-0.5B-Instruct, using HaluEval's faithful responses as chosen and hallucinated responses as rejected, creating 21K preference pairs. Training runs for 1 epoch with learning rate 5e-6 and beta=0.1.

The results are striking. The base Qwen2.5-0.5B model produces hallucinated responses for 85.5% of held-out test prompts, as scored by the DeBERTa detector. After DPO training, the hallucination rate drops to 37.7%, a 55.9% relative reduction. The mean detector probability falls from 0.816 to 0.293, and the probability distribution shifts from concentration near P(hall)=1.0 toward P(hall)=0. Two caveats apply: the detector and the DPO preference signal share supervision from HaluEval pairs, making this a co-evaluation rather than a fully held-out test, and the detector serves as a consistent automated evaluation signal rather than a gold-standard verdict.

Why Cross-Domain Transfer Fails and How to Fix It

The cross-domain experiments expose a real limitation. Applying the HaluEval-trained detector zero-shot to SciFact, a biomedical claim verification dataset with 1,109 scientific claims and 5,183 abstracts, yields F1=0.517 and AUROC=0.515, barely above chance. The model predicts nearly all scientific claims as hallucinated because biomedical claim-evidence pairs use technical vocabulary, hedged language, and citation-grounded reasoning that the source-domain training never encountered.

Three adaptation strategies are compared. Fine-tuning DeBERTa-v3 on SciFact alone (Config A) achieves F1=0.488 and AUROC=0.582. Sequential transfer from HaluEval to SciFact (Config C) improves AUROC to 0.610 but F1 to only 0.533. Fine-tuning PubMedBERT, pre-trained on PubMed abstracts and PMC full-text, on SciFact (Config B) achieves the best results: F1=0.627 and AUROC=0.808. The ranking domain-matched pre-training over source-task transfer over in-domain training alone over zero-shot demonstrates that the dominant signal in cross-domain hallucination detection is the pre-training corpus, not the fine-tuning data. All configurations remain below 0.7 F1 with only 484 training examples, suggesting that more domain-specific data would further close the gap.

What This Means for Production Hallucination Detection

The practical takeaways are concrete. MC Dropout with 20 passes provides a cheap, training-free accuracy boost of nearly 2 points and an uncertainty signal that identifies ambiguous cases. Around 5,000 labeled examples suffice for a usable detector in a new domain. The DeBERTa-v3 backbone with disentangled attention handles the NLI formulation well, and temperature scaling stabilizes probability estimates across domains. For mitigation, DPO with the detector in the loop cuts hallucination rates by more than half, though the co-evaluation caveat means independent verification matters.

The cross-domain results are a cautionary note for anyone assuming a general-domain detector transfers to specialized text. PubMedBERT's strong performance suggests that matching the pre-training corpus to the target domain is more effective than fine-tuning alone or sequential transfer. For teams deploying hallucination detection in medicine, law, or finance, the path forward is clear: start with a domain-matched pre-trained model, fine-tune on a few thousand domain-specific labeled examples, and use MC Dropout to get calibrated uncertainty estimates for the cases that matter most.

Read the paper on arXiv