Explainable Artificial Intelligence has become essential for trustworthy deployment of deep learning systems, yet a fundamental challenge persists: different attribution methods often produce contradictory explanations for the same model prediction. This disagreement problem undermines confidence in model interpretations, particularly in high-stakes domains like medicine where understanding reasoning matters as much as accuracy. A comparative explainability framework is presented to audit DeBERTa-v3 under zero-shot classification of medical abstracts, directly addressing this issue by systematically evaluating and quantifying explanatory agreement across five diverse attribution methods.
The work implements a natural language inference engine over a Medical Abstracts corpus enriched with five hypotheses per diagnostic category, using a balanced sample of one thousand texts per class. Five explanation methods are compared: SHAP and LIME as model-agnostic approaches, occlusion and Input x Gradient as deep-learning-specific approaches, and Attention x Gradient as a transformer-specific approach. Explanations are standardized through top-token attribution, and pairwise agreement is quantified using the Jaccard index. High predictive accuracy is achieved across well-defined clinical domains, whereas performance degrades under high semantic ambiguity. Explanatory stability directly mirrors predictive certainty, exhibiting strong convergence in univalent categories and a marked drop under diagnostic uncertainty. Furthermore, qualitative error auditing uncovers three systemic failure mechanisms: lexical hypersensitivity, semantic overlap, and loss of attribution coherence. The results support the combined use of several explanation methods and quantitative agreement metrics when auditing transformer-based models in medical text classification, and suggest prioritizing specific clinical ontologies over broad diagnostic labels.
The disagreement problem in medical explainability
Explainable AI has matured rapidly, with methods ranging from gradient-based attribution to game-theoretic Shapley values. However, a growing body of work highlights that no single method reliably reveals model behavior across diverse inputs and architectures. In medical natural language processing, this problem is acute: a clinician relying on a single attribution map might misinterpret why a model flagged a particular abstract, potentially overlooking critical reasoning paths or, conversely, being misled by spurious correlations. The disagreement among explanations is not merely academic—it has practical consequences for model debugging, regulatory compliance, and clinical trust.
Prior studies have noted that attribution methods can diverge even on identical inputs. Some methods are sensitive to token frequency rather than causal influence, while others may overemphasize rare but distinctive words. In the zero-shot setting, where models are tested on categories they were not explicitly trained on, these divergences can be especially pronounced. The present work systematically investigates these phenomena using DeBERTa-v3, a disentangled attention architecture that has become a strong baseline for text classification tasks.
Framework design and data construction
The framework operates on a curated corpus of medical abstracts with fine-grained diagnostic labels. For each class, five enriched hypotheses are defined, capturing common clinical reasoning patterns such as presence of specific disease markers, exclusion of differential diagnoses, and progression indicators. A balanced sample of one thousand texts per class ensures that no single category dominates the evaluation, while also providing sufficient material for statistical comparison across explanation methods.
The core innovation of this work lies not in proposing a new attribution method, but in creating a rigorous comparative pipeline. Each method—SHAP, LIME, occlusion, Input x Gradient, and Attention x Gradient—is applied to the same model predictions, and their outputs are standardized through top-token attribution. This normalization allows direct comparison: instead of comparing raw importance scores or heatmap intensities, the analysis focuses on which individual tokens each method identifies as most influential for a given prediction.
Pairwise agreement between methods is then quantified using the Jaccard index, which measures the overlap between the sets of top-attributed tokens. A Jaccard index of 1 indicates perfect agreement (the same tokens selected by both methods), while 0 indicates no overlap. By computing these indices across all test instances, the authors generate a distribution of agreement scores that reveals patterns of convergence and divergence.
Results: agreement patterns across methods
The results reveal a clear hierarchy of methodological consistency. SHAP and LIME, as model-agnostic approaches, show moderate agreement with each other but significantly lower overlap with the deep-learning-specific methods. Occlusion and Input x Gradient, both operating at the pixel or token level, converge more frequently with each other than with the model-agnostic approaches. Attention x Gradient, being transformer-specific, exhibits a unique pattern: it agrees more often with other token-level methods while showing distinct behavior regarding long-range dependencies.
Perhaps the most striking finding is the relationship between explanatory stability and predictive certainty. In categories where the model achieves high confidence—what the authors term "univalent" categories—explanations from all methods tend to converge on a small set of diagnostic keywords. For example, in abstracts clearly identifying a specific pathogen, multiple methods consistently highlight the organism name, the associated clinical syndrome, and pathognomonic terminology. This convergence suggests that when the model's prediction is unambiguous, the underlying linguistic signals are also unambiguous, and different explanation frameworks happen to capture the same cues.
Conversely, under high semantic ambiguity—such as abstracts describing overlapping conditions, indeterminate findings, or rare disease presentations—explanatory stability collapses. The Jaccard indices drop markedly, and different methods latch onto different tokens: some focus on foreground disease terms, others on negation markers, and still others on contextual phrasing that qualifies the diagnosis. This degradation is not random; it systematically aligns with the model's own declining confidence, as measured by prediction probability.
Qualitative error auditing and systemic failure modes
Beyond quantitative agreement metrics, the authors conduct qualitative error auditing, examining cases where explanations fail to align with intuitive clinical reasoning. Three systemic failure mechanisms emerge consistently:
Lexical hypersensitivity: certain methods over-index on rare or distinctive words that appear in the training data but may not be causally relevant to the prediction. In medical contexts, this can manifest as disproportionate attention to study-level phrasing, grant numbering, or institutional identifiers that coincidentally correlate with certain diagnostic categories in the test split.
Semantic overlap: when abstracts discuss multiple conditions or use terminology that spans diagnostic categories, attribution methods struggle to isolate the most relevant signals. This is particularly problematic in zero-shot settings where the model has not seen explicit examples of certain condition combinations during training.
Loss of attribution coherence: some methods produce scattered or fragmented attributions that fail to form a coherent rationale. An explanation that equally weights a disease name, a treatment mention, and a demographic descriptor, for instance, may be technically valid per the method's mathematics but practically unhelpful for understanding the model's decision.
These failure modes are not attributable to any single explanation method; rather, they reflect fundamental tensions between different philosophies of what constitutes a "reasonable" explanation. Methods optimized for sensitivity may produce noisy attributions, while methods optimized for specificity may miss subtle but important signals.
Practical implications and recommendations
The framework's most immediate contribution is a set of evidence-based recommendations for auditing transformer-based models in medical text classification. The authors argue against relying on a single explanation method, noting that the complementary strengths of the compared approaches provide a more complete picture when used together. SHAP offers model-agnostic transparency, LIME provides local perturbation-based insights, occlusion and Input x Gradient capture concrete token-level influences, and Attention x Gradient leverages the architectural specifics of transformer attention patterns.
Perhaps more importantly, the paper suggests prioritizing specific clinical ontologies over broad diagnostic labels. A model classifying abstracts into high-level categories such as "infectious disease" or "neoplasm" may receive superficially coherent explanations, but these may mask finer-grained reasoning about specific pathogens, tumor subtypes, or staging criteria. By evaluating explanations at the ontology level—e.g., assessing whether the model correctly attends to influenza-specific markers when classifying an abstract as influenza—the framework enables more granular quality assurance.
The quantitative agreement metrics, particularly the Jaccard index distributions, serve as diagnostic tools themselves. A sudden drop in average agreement across methods for a particular subset of test instances can signal distributional shift, annotation artifacts, or model degradation warranting further investigation. In a production setting, monitoring these metrics over time could provide an early warning system for changing model behavior.
Limitations and directions for future work
The authors acknowledge several limitations. The evaluation is confined to zero-shot classification of medical abstracts; other medical NLP tasks—such as clinical note generation, question answering, or radiology report analysis—may exhibit different explanation patterns. The corpus, while balanced across classes, represents a specific subset of medical literature, and generalization to other domains (e., electronic health records, patient-facing materials) remains an open question. Additionally, the framework focuses on attribution-level explainability; other forms of explanation, such as counterfactual reasoning or causal intervention, were outside its scope.
Future work could extend the comparative pipeline to other transformer architectures, including encoder-only models like DeBERTa and decoder-only or encoder-decoder models prevalent in medical generative AI. Incorporating human clinician evaluations of explanation quality would also bridge the gap between metric-level agreement and clinically meaningful interpretations. Finally, investigating dynamic method selection—choosing the most appropriate explanation technique based on the specific prediction context—could further improve the practical utility of XAI pipelines.
Overall, this work provides a structured, reproducible framework for understanding and comparing explanation behaviors in medical NLP. By quantifying agreement, identifying failure modes, and offering evidence-based recommendations, it advances the state of the art in making transformer-based models more transparent and trustworthy for clinical applications.